[
https://issues.apache.org/jira/browse/BEAM-8619?focusedWorklogId=347836&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-347836
]
ASF GitHub Bot logged work on BEAM-8619:
----------------------------------------
Author: ASF GitHub Bot
Created on: 22/Nov/19 00:37
Start Date: 22/Nov/19 00:37
Worklog Time Spent: 10m
Work Description: lukecwik commented on pull request #10126: [BEAM-8619]
Tear down the DoFns upon the control service termination …
URL: https://github.com/apache/beam/pull/10126#discussion_r349385020
##########
File path:
sdks/java/harness/src/main/java/org/apache/beam/fn/harness/control/ProcessBundleHandler.java
##########
@@ -429,6 +434,22 @@ void release(String bundleDescriptorId, BundleProcessor
bundleProcessor) {
bundleProcessor.reset();
cachedBundleProcessors.get(bundleDescriptorId).add(bundleProcessor);
}
+
+ /** Shutdown all the cached {@link BundleProcessor}s, running the
tearDown() functions. */
+ void shutdown() throws Exception {
+ for (ConcurrentLinkedQueue<BundleProcessor> bundleProcessors :
+ cachedBundleProcessors.values()) {
+ for (BundleProcessor bundleProcessor : bundleProcessors) {
+ // Need to reverse this since we want to call teardown in
topological order.
Review comment:
It is not necessary to reverse the list since teardown can't produce output
and should not meaningfully impact other instances. Its the same with
startInstance where the order doesn't matter.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 347836)
Time Spent: 1.5h (was: 1h 20m)
> Tear down the DoFns upon the control service termination in Java SDK harness
> ----------------------------------------------------------------------------
>
> Key: BEAM-8619
> URL: https://issues.apache.org/jira/browse/BEAM-8619
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-harness
> Affects Versions: 2.18.0
> Reporter: sunjincheng
> Assignee: sunjincheng
> Priority: Major
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Per the discussion in the ML, the detail can be found [1], the teardown of
> DoFns should be supported in the portability framework. It happens at two
> places:
> 1) Upon the control service termination
> 2) Tear down the unused DoFns periodically
> The aim of this JIRA is to add support for teardown the DoFns upon the
> control service termination in Java SDK harness.
> [1]
> https://lists.apache.org/thread.html/0c4a4cf83cf2e35c3dfeb9d906e26cd82d3820968ba6f862f91739e4@%3Cdev.beam.apache.org%3E
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
