[
https://issues.apache.org/jira/browse/BEAM-6672?focusedWorklogId=199044&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-199044
]
ASF GitHub Bot logged work on BEAM-6672:
----------------------------------------
Author: ASF GitHub Bot
Created on: 15/Feb/19 01:03
Start Date: 15/Feb/19 01:03
Worklog Time Spent: 10m
Work Description: Ardagan commented on pull request #7847: [BEAM-6672]
Add the StateRequestHandlerImpl and test
URL: https://github.com/apache/beam/pull/7847#discussion_r257068934
##########
File path:
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/control/StateRequestHandlerImpl.java
##########
@@ -55,13 +58,15 @@ public
StateRequestHandlerImpl(DataflowExecutionContext.DataflowStepContext ctxt
}
public void finish() {
- userStateData.clear();
+ for (Map.Entry<BeamFnApi.StateKey, BagState<ByteString>> entry :
userStateData.entrySet()) {
Review comment:
So I dug a bit into ctx.stateInernals().stat() code. Seems that user
BagState is already cached inside
[StateTable](https://github.com/apache/beam/blob/master/runners/core-java/src/main/java/org/apache/beam/runners/core/StateTable.java#L48)..
If my understanding is correct, [this
line](https://github.com/apache/beam/pull/7847/files/dad2b2d5b9b616ddfc1461f074e3bf67a41b7eaa..6d79ce0a6ea3ac396fefad624f55e3f15351bb02#diff-112970b4df06018132273f194990d4ebR79)
eventually leads
[here](https://github.com/apache/beam/blob/master/runners/core-java/src/main/java/org/apache/beam/runners/core/StateTable.java#L48).
Here you can see that state is actually persist in state table and when
BagUserState is missing in userStateData, but present in StateTable, you will
insert new value into userStateData with non-empty state.
This way when you change from userStateData.clear() to the loop, you do not
only clean userStateData.clear(), but also clear the state in StateTable
utilized in ctxt.stateInternals().state
Given this information, can you elaborate a bit more on the intent of this
class and it's further usage?
Can you elaborate as well on what you're trying to clean in finish() method:
Is it actually clear State completely from worker, or only forget about it in
StateRequestHandlerImpl hash map?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 199044)
Time Spent: 2.5h (was: 2h 20m)
> Make bundle execution with ExecutableStage support user states
> --------------------------------------------------------------
>
> Key: BEAM-6672
> URL: https://issues.apache.org/jira/browse/BEAM-6672
> Project: Beam
> Issue Type: Improvement
> Components: runner-dataflow
> Reporter: Sam Rohde
> Assignee: Sam Rohde
> Priority: Major
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
