[
https://issues.apache.org/jira/browse/BEAM-8333?focusedWorklogId=325316&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-325316
]
ASF GitHub Bot logged work on BEAM-8333:
----------------------------------------
Author: ASF GitHub Bot
Created on: 08/Oct/19 20:30
Start Date: 08/Oct/19 20:30
Worklog Time Spent: 10m
Work Description: pabloem commented on issue #9746: [BEAM-8333] Adding
lull logging for SDK harness
URL: https://github.com/apache/beam/pull/9746#issuecomment-539690884
This logs something like this:
```
WARNING:root:There has been a processing lull of over 0.45 seconds in state
process-msecs in step Map(sleep) . Traceback:
File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap
self._bootstrap_inner()
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3.6/concurrent/futures/thread.py", line 69, in _worker
work_item.run()
File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
result = self.fn(*self.args, **self.kwargs)
File
"/usr/local/google/home/pabloem/codes/log-lull-sdk/sdks/python/apache_beam/runners/worker/sdk_worker.py",
line 203, in task
self._execute(lambda: worker.do_instruction(work), work)
File
"/usr/local/google/home/pabloem/codes/log-lull-sdk/sdks/python/apache_beam/runners/worker/sdk_worker.py",
line 170, in _execute
response = task()
File
"/usr/local/google/home/pabloem/codes/log-lull-sdk/sdks/python/apache_beam/runners/worker/sdk_worker.py",
line 203, in <lambda>
self._execute(lambda: worker.do_instruction(work), work)
File
"/usr/local/google/home/pabloem/codes/log-lull-sdk/sdks/python/apache_beam/runners/worker/sdk_worker.py",
line 360, in do_instruction
request.instruction_id)
File
"/usr/local/google/home/pabloem/codes/log-lull-sdk/sdks/python/apache_beam/runners/worker/sdk_worker.py",
line 386, in process_bundle
bundle_processor.process_bundle(instruction_id))
File
"/usr/local/google/home/pabloem/codes/log-lull-sdk/sdks/python/apache_beam/runners/worker/bundle_processor.py",
line 663, in process_bundle
data.transform_id].process_encoded(data.data)
File
"/usr/local/google/home/pabloem/codes/log-lull-sdk/sdks/python/apache_beam/runners/worker/bundle_processor.py",
line 143, in process_encoded
self.output(decoded_value)
File
"/usr/local/google/home/pabloem/codes/log-lull-sdk/sdks/python/apache_beam/transforms/core.py",
line 1356, in <lambda>
wrapper = lambda x, *args, **kwargs: [fn(x, *args, **kwargs)]
.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 325316)
Remaining Estimate: 0h
Time Spent: 10m
> Python SDK Worker should log lulls with progress-reporting thread
> -----------------------------------------------------------------
>
> Key: BEAM-8333
> URL: https://issues.apache.org/jira/browse/BEAM-8333
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-harness
> Reporter: Pablo Estrada
> Assignee: Pablo Estrada
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
