[
https://issues.apache.org/jira/browse/BEAM-5633?focusedWorklogId=153221&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-153221
]
ASF GitHub Bot logged work on BEAM-5633:
----------------------------------------
Author: ASF GitHub Bot
Created on: 10/Oct/18 17:23
Start Date: 10/Oct/18 17:23
Worklog Time Spent: 10m
Work Description: aaltay commented on a change in pull request #6593:
[BEAM-5633] Adds reconnect logic to python logging client
URL: https://github.com/apache/beam/pull/6593#discussion_r224168930
##########
File path: sdks/python/apache_beam/runners/worker/log_handler.py
##########
@@ -106,7 +120,19 @@ def _write_log_entries(self):
if log_entries:
yield beam_fn_api_pb2.LogEntry.List(log_entries=log_entries)
- def _read_log_control_messages(self, log_control_iterator):
- # TODO(vikasrk): Handle control messages.
- for _ in log_control_iterator:
- pass
+ def _read_log_control_messages(self):
+ while True:
+ log_control_iterator = self.connect()
+ if self._dropped_logs > 0:
+ logging.warn("Dropped %d logs while logging client disconnected",
Review comment:
Thank you for the thoughts. Some options I can think of are:
- One option is, just acknowledge the issue add a comment along the lines of
this is best effort.
- Another option is hold the lock, drop the last item from the queue to
ensure there is at least one spot, and insert this new item.
- If possible we can try to hold the lock, extend the queue capacity by one
and log then.
I agree that logging to stderr is not useful.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 153221)
Time Spent: 3.5h (was: 3h 20m)
> Python SDK harness logging client failure
> -----------------------------------------
>
> Key: BEAM-5633
> URL: https://issues.apache.org/jira/browse/BEAM-5633
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-harness
> Reporter: Thomas Weise
> Assignee: Micah Wylde
> Priority: Major
> Labels: portability-flink
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> After running a test with synthetic source for a few minutes, the logging
> client fails and all subsequent log output is not forwarded to the runner.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
