[
https://issues.apache.org/jira/browse/BEAM-11144?focusedWorklogId=507301&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-507301
]
ASF GitHub Bot logged work on BEAM-11144:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 03/Nov/20 19:43
Start Date: 03/Nov/20 19:43
Worklog Time Spent: 10m
Work Description: scwhittle commented on a change in pull request #13221:
URL: https://github.com/apache/beam/pull/13221#discussion_r516913548
##########
File path:
runners/core-java/src/main/java/org/apache/beam/runners/core/triggers/AfterEachStateMachine.java
##########
@@ -48,6 +48,13 @@ private AfterEachStateMachine(List<TriggerStateMachine>
subTriggers) {
checkArgument(subTriggers.size() > 1);
}
+ @Override
+ public void prefetchOnElement(PrefetchContext c) {
+ for (ExecutableTriggerStateMachine subTrigger : c.trigger().subTriggers())
{
Review comment:
Yeah when I embarked on this I was thinking also it might be possible to
do better but it didn't pan out.
One idea I had was adding a isCached() method on the state objects that
would indicate if the value was local (cached, in-memory, etc) and didn't
require a fetch. In such cases we could trim the trigger state to prefetch by
looking at the closed triggers. However if everything we actually needed was
local, we've only called readLater on something we never actually read and thus
never issue a fetch anyway. And if we do have to issue a fetch for anything,
fetching an additional unused tag to it is likely not much more overhead.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 507301)
Time Spent: 1h 10m (was: 1h)
> TriggerStateMachine.prefetchOnElement and other prefetch methods use
> incorrect state for subtriggers
> ----------------------------------------------------------------------------------------------------
>
> Key: BEAM-11144
> URL: https://issues.apache.org/jira/browse/BEAM-11144
> Project: Beam
> Issue Type: Bug
> Components: runner-core
> Reporter: Sam Whittle
> Assignee: Sam Whittle
> Priority: P2
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> prefetchOnElement takes a single StateAccessor corresponding to the root
> trigger. It uses that state accessor to perform prefetches for the
> subtrigger state. However that will use the root trigger namespace for the
> tags in the subtriggers. This makes prefetching ineffective, introducing
> possibly an additional round-trip once the data is actually read.
> https://github.com/apache/beam/blob/68d6c8e6243b1d8f392840273f886276e2a8baff/runners/core-java/src/main/java/org/apache/beam/runners/core/triggers/TriggerStateMachine.java#L296
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
