See
<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/2/display/redirect>
------------------------------------------
[...truncated 689.91 KB...]
"output_info": [
{
"encoding": {
"@type": "kind:windowed_value",
"component_encodings": [
{
"@type":
"FastPrimitivesCoder$eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/",
"component_encodings": [
{
"@type":
"FastPrimitivesCoder$eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/",
"component_encodings": []
},
{
"@type":
"FastPrimitivesCoder$eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/",
"component_encodings": []
}
],
"is_pair_like": true
},
{
"@type": "kind:global_window"
}
],
"is_wrapper": true
},
"output_name": "out",
"user_name": "ParDo(CounterDoFn).out"
}
],
"parallel_input": {
"@type": "OutputReference",
"output_name": "out",
"step_name": "s1"
},
"serialized_fn": "<string of 1168 bytes>",
"user_name": "ParDo(CounterDoFn)"
}
}
],
"type": "JOB_TYPE_BATCH"
}
root: INFO: Create job: <Job
createTime: u'2018-05-15T23:50:14.593946Z'
currentStateTime: u'1970-01-01T00:00:00Z'
id: u'2018-05-15_16_50_13-8192291641162623762'
location: u'us-central1'
name: u'beamapp-jenkins-0515235012-609134'
projectId: u'apache-beam-testing'
stageStates: []
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
root: INFO: Created job with id: [2018-05-15_16_50_13-8192291641162623762]
root: INFO: To access the Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-15_16_50_13-8192291641162623762?project=apache-beam-testing
root: INFO: Job 2018-05-15_16_50_13-8192291641162623762 is in state
JOB_STATE_RUNNING
root: INFO: 2018-05-15T23:50:13.414Z: JOB_MESSAGE_WARNING: Job
2018-05-15_16_50_13-8192291641162623762 might autoscale up to 1000 workers.
root: INFO: 2018-05-15T23:50:13.447Z: JOB_MESSAGE_DETAILED: Autoscaling is
enabled for job 2018-05-15_16_50_13-8192291641162623762. The number of workers
will be between 1 and 1000.
root: INFO: 2018-05-15T23:50:13.482Z: JOB_MESSAGE_DETAILED: Autoscaling was
automatically enabled for job 2018-05-15_16_50_13-8192291641162623762.
root: INFO: 2018-05-15T23:50:16.278Z: JOB_MESSAGE_DETAILED: Checking required
Cloud APIs are enabled.
root: INFO: 2018-05-15T23:50:16.598Z: JOB_MESSAGE_DETAILED: Checking
permissions granted to controller Service Account.
root: INFO: 2018-05-15T23:50:17.243Z: JOB_MESSAGE_BASIC: Worker configuration:
n1-standard-1 in us-central1-b.
root: INFO: 2018-05-15T23:50:17.740Z: JOB_MESSAGE_DETAILED: Expanding
CoGroupByKey operations into optimizable parts.
root: INFO: 2018-05-15T23:50:17.771Z: JOB_MESSAGE_DETAILED: Expanding
GroupByKey operations into optimizable parts.
root: INFO: 2018-05-15T23:50:17.803Z: JOB_MESSAGE_DETAILED: Lifting
ValueCombiningMappingFns into MergeBucketsMappingFns
root: INFO: 2018-05-15T23:50:17.838Z: JOB_MESSAGE_DEBUG: Annotating graph with
Autotuner information.
root: INFO: 2018-05-15T23:50:17.883Z: JOB_MESSAGE_DETAILED: Fusing adjacent
ParDo, Read, Write, and Flatten operations
root: INFO: 2018-05-15T23:50:17.923Z: JOB_MESSAGE_DETAILED: Fusing consumer
ParDo(CounterDoFn) into Read
root: INFO: 2018-05-15T23:50:17.955Z: JOB_MESSAGE_DEBUG: Workflow config is
missing a default resource spec.
root: INFO: 2018-05-15T23:50:17.992Z: JOB_MESSAGE_DEBUG: Adding StepResource
setup and teardown to workflow graph.
root: INFO: 2018-05-15T23:50:18.016Z: JOB_MESSAGE_DEBUG: Adding workflow start
and stop steps.
root: INFO: 2018-05-15T23:50:18.049Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
root: INFO: 2018-05-15T23:50:18.208Z: JOB_MESSAGE_DEBUG: Executing wait step
start3
root: INFO: 2018-05-15T23:50:18.301Z: JOB_MESSAGE_BASIC: Executing operation
Read+ParDo(CounterDoFn)
root: INFO: 2018-05-15T23:50:18.338Z: JOB_MESSAGE_DEBUG: Starting worker pool
setup.
root: INFO: 2018-05-15T23:50:18.377Z: JOB_MESSAGE_BASIC: Starting 1 workers in
us-central1-b...
root: INFO: 2018-05-15T23:50:25.660Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised
the number of workers to 0 based on the rate of progress in the currently
running step(s).
root: INFO: 2018-05-15T23:50:36.391Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised
the number of workers to 1 based on the rate of progress in the currently
running step(s).
root: INFO: 2018-05-15T23:52:38.218Z: JOB_MESSAGE_DETAILED: Workers have
started successfully.
root: INFO: 2018-05-15T23:55:12.841Z: JOB_MESSAGE_ERROR: Traceback (most recent
call last):
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py",
line 609, in do_work
work_executor.execute()
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py",
line 157, in execute
op.start()
File "apache_beam/runners/worker/operations.py", line 351, in
apache_beam.runners.worker.operations.DoOperation.start
def start(self):
File "apache_beam/runners/worker/operations.py", line 352, in
apache_beam.runners.worker.operations.DoOperation.start
with self.scoped_start_state:
File "apache_beam/runners/worker/operations.py", line 357, in
apache_beam.runners.worker.operations.DoOperation.start
pickler.loads(self.spec.serialized_fn))
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/internal/pickler.py", line
230, in loads
return dill.loads(s)
File "/usr/local/lib/python2.7/dist-packages/dill/dill.py", line 277, in loads
return load(file)
File "/usr/local/lib/python2.7/dist-packages/dill/dill.py", line 266, in load
obj = pik.load()
File "/usr/lib/python2.7/pickle.py", line 864, in load
dispatch[key](self)
File "/usr/lib/python2.7/pickle.py", line 1139, in load_reduce
value = func(*args)
File "/usr/local/lib/python2.7/dist-packages/dill/dill.py", line 767, in
_import_module
return getattr(__import__(module, None, None, [obj]), obj)
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/transforms/ptransform_test.py",
line 29, in <module>
import hamcrest as hc
ImportError: No module named hamcrest
root: INFO: 2018-05-15T23:55:16.164Z: JOB_MESSAGE_ERROR: Traceback (most recent
call last):
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py",
line 609, in do_work
work_executor.execute()
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py",
line 157, in execute
op.start()
File "apache_beam/runners/worker/operations.py", line 351, in
apache_beam.runners.worker.operations.DoOperation.start
def start(self):
File "apache_beam/runners/worker/operations.py", line 352, in
apache_beam.runners.worker.operations.DoOperation.start
with self.scoped_start_state:
File "apache_beam/runners/worker/operations.py", line 357, in
apache_beam.runners.worker.operations.DoOperation.start
pickler.loads(self.spec.serialized_fn))
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/internal/pickler.py", line
230, in loads
return dill.loads(s)
File "/usr/local/lib/python2.7/dist-packages/dill/dill.py", line 277, in loads
return load(file)
File "/usr/local/lib/python2.7/dist-packages/dill/dill.py", line 266, in load
obj = pik.load()
File "/usr/lib/python2.7/pickle.py", line 864, in load
dispatch[key](self)
File "/usr/lib/python2.7/pickle.py", line 1139, in load_reduce
value = func(*args)
File "/usr/local/lib/python2.7/dist-packages/dill/dill.py", line 767, in
_import_module
return getattr(__import__(module, None, None, [obj]), obj)
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/transforms/ptransform_test.py",
line 29, in <module>
import hamcrest as hc
ImportError: No module named hamcrest
root: INFO: 2018-05-15T23:55:19.517Z: JOB_MESSAGE_ERROR: Traceback (most recent
call last):
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py",
line 609, in do_work
work_executor.execute()
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py",
line 157, in execute
op.start()
File "apache_beam/runners/worker/operations.py", line 351, in
apache_beam.runners.worker.operations.DoOperation.start
def start(self):
File "apache_beam/runners/worker/operations.py", line 352, in
apache_beam.runners.worker.operations.DoOperation.start
with self.scoped_start_state:
File "apache_beam/runners/worker/operations.py", line 357, in
apache_beam.runners.worker.operations.DoOperation.start
pickler.loads(self.spec.serialized_fn))
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/internal/pickler.py", line
230, in loads
return dill.loads(s)
File "/usr/local/lib/python2.7/dist-packages/dill/dill.py", line 277, in loads
return load(file)
File "/usr/local/lib/python2.7/dist-packages/dill/dill.py", line 266, in load
obj = pik.load()
File "/usr/lib/python2.7/pickle.py", line 864, in load
dispatch[key](self)
File "/usr/lib/python2.7/pickle.py", line 1139, in load_reduce
value = func(*args)
File "/usr/local/lib/python2.7/dist-packages/dill/dill.py", line 767, in
_import_module
return getattr(__import__(module, None, None, [obj]), obj)
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/transforms/ptransform_test.py",
line 29, in <module>
import hamcrest as hc
ImportError: No module named hamcrest
root: INFO: 2018-05-15T23:55:22.863Z: JOB_MESSAGE_ERROR: Traceback (most recent
call last):
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py",
line 609, in do_work
work_executor.execute()
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py",
line 157, in execute
op.start()
File "apache_beam/runners/worker/operations.py", line 351, in
apache_beam.runners.worker.operations.DoOperation.start
def start(self):
File "apache_beam/runners/worker/operations.py", line 352, in
apache_beam.runners.worker.operations.DoOperation.start
with self.scoped_start_state:
File "apache_beam/runners/worker/operations.py", line 357, in
apache_beam.runners.worker.operations.DoOperation.start
pickler.loads(self.spec.serialized_fn))
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/internal/pickler.py", line
230, in loads
return dill.loads(s)
File "/usr/local/lib/python2.7/dist-packages/dill/dill.py", line 277, in loads
return load(file)
File "/usr/local/lib/python2.7/dist-packages/dill/dill.py", line 266, in load
obj = pik.load()
File "/usr/lib/python2.7/pickle.py", line 864, in load
dispatch[key](self)
File "/usr/lib/python2.7/pickle.py", line 1139, in load_reduce
value = func(*args)
File "/usr/local/lib/python2.7/dist-packages/dill/dill.py", line 767, in
_import_module
return getattr(__import__(module, None, None, [obj]), obj)
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/transforms/ptransform_test.py",
line 29, in <module>
import hamcrest as hc
ImportError: No module named hamcrest
root: INFO: 2018-05-15T23:55:22.900Z: JOB_MESSAGE_DEBUG: Executing failure step
failure2
root: INFO: 2018-05-15T23:55:22.944Z: JOB_MESSAGE_ERROR: Workflow failed.
Causes: S01:Read+ParDo(CounterDoFn) failed., A work item was attempted 4 times
without success. Each time the worker eventually lost contact with the service.
The work item was attempted on:
beamapp-jenkins-051523501-05151650-9j6z-harness-c4gj,
beamapp-jenkins-051523501-05151650-9j6z-harness-c4gj,
beamapp-jenkins-051523501-05151650-9j6z-harness-c4gj,
beamapp-jenkins-051523501-05151650-9j6z-harness-c4gj
root: INFO: 2018-05-15T23:55:23.080Z: JOB_MESSAGE_DETAILED: Cleaning up.
root: INFO: 2018-05-15T23:55:23.146Z: JOB_MESSAGE_DEBUG: Starting worker pool
teardown.
root: INFO: 2018-05-15T23:55:23.166Z: JOB_MESSAGE_BASIC: Stopping worker pool...
root: INFO: 2018-05-15T23:57:29.025Z: JOB_MESSAGE_DETAILED: Autoscaling:
Resized worker pool from 1 to 0.
root: INFO: 2018-05-15T23:57:29.065Z: JOB_MESSAGE_DETAILED: Autoscaling: Would
further reduce the number of workers but reached the minimum number allowed for
the job.
root: INFO: 2018-05-15T23:57:29.110Z: JOB_MESSAGE_DEBUG: Tearing down pending
resources...
root: INFO: Job 2018-05-15_16_50_13-8192291641162623762 is in state
JOB_STATE_FAILED
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
XML:
<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 16 tests in 1917.120s
FAILED (errors=6)
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-15_16_43_15-13225726923727899038?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-15_16_51_03-7825513605277141009?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-15_16_59_16-10768468270593702170?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-15_17_06_08-8146974819649373710?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-15_16_43_16-108144182704794975?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-15_16_51_08-8910811707149951895?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-15_16_59_05-15186441445262452879?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-15_17_05_48-2023310276940020703?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-15_16_43_16-11547189168163978248?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-15_16_50_29-13767611942966351942?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-15_16_57_16-508155604053594607?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-15_17_04_23-5340474747907776824?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-15_16_43_16-11743476521765802298?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-15_16_50_13-8192291641162623762?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-15_16_57_40-6059369709558260006?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-15_17_04_38-4887584052602197210?project=apache-beam-testing.
> Task :beam-sdks-python:validatesRunnerTests FAILED
:beam-sdks-python:validatesRunnerTests (Thread[Task worker for ':' Thread
10,5,main]) completed. Took 31 mins 58.603 secs.
FAILURE: Build failed with an exception.
* Where:
Build file
'<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/build.gradle'>
line: 201
* What went wrong:
Execution failed for task ':beam-sdks-python:validatesRunnerTests'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 32m 16s
3 actionable tasks: 3 executed
Publishing build scan...
https://gradle.com/s/kaexqvwsk2vpu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
