[
https://issues.apache.org/jira/browse/BEAM-6777?focusedWorklogId=211301&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-211301
]
ASF GitHub Bot logged work on BEAM-6777:
----------------------------------------
Author: ASF GitHub Bot
Created on: 11/Mar/19 21:17
Start Date: 11/Mar/19 21:17
Worklog Time Spent: 10m
Work Description: rohdesamuel commented on pull request #8012:
[BEAM-6777] Add HealthDaemon and tests
URL: https://github.com/apache/beam/pull/8012#discussion_r264434288
##########
File path: sdks/python/apache_beam/runners/worker/health_daemon.py
##########
@@ -0,0 +1,121 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+from __future__ import absolute_import
+
+import errno
+import http.client
+import logging
+import socket
+import time
+from builtins import object
+
+
+class HealthDaemon(object):
+ """Sends periodic HTTP PUT /sdk requests to the health server.
+
+ The purpose of this class is to communicate to the health server that this
+ SDK Harness is alive. If this SDK Harness does not communicate to the health
+ server after a configured amount of time, the health server will restart the
+ container.
+
+ Expected Usage:
+ # The HealthDaemon is expected to spin forever, start it on a separate
+ # thread.
+ health_thread = threading.Thread(target=HealthDaemon(8080).start)
+
+ # Automatically kill the thread when the program exists.
Review comment:
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 211301)
Time Spent: 1h 10m (was: 1h)
> SDK Harness Resilience
> ----------------------
>
> Key: BEAM-6777
> URL: https://issues.apache.org/jira/browse/BEAM-6777
> Project: Beam
> Issue Type: Improvement
> Components: runner-dataflow
> Reporter: Sam Rohde
> Assignee: Sam Rohde
> Priority: Major
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> If the Python SDK Harness crashes in any way (user code exception, OOM, etc)
> the job will hang and waste resources. The fix is to add a daemon in the SDK
> Harness and Runner Harness to communicate with Dataflow to restart the VM
> when stuckness is detected.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
