[
https://issues.apache.org/jira/browse/BEAM-6608?focusedWorklogId=195261&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-195261
]
ASF GitHub Bot logged work on BEAM-6608:
----------------------------------------
Author: ASF GitHub Bot
Created on: 06/Feb/19 17:39
Start Date: 06/Feb/19 17:39
Worklog Time Spent: 10m
Work Description: mxm commented on issue #7748: [BEAM-6608] Prepare
to-be-staged files early enough in Flink Runner
URL: https://github.com/apache/beam/pull/7748#issuecomment-461115767
>In the prior change, the preparation of staged files is deliberately (and,
I think, correctly) moved after a graph rewrite. I'm not sure that can really
change the set of staged files in practice, but in theory it certainly could. I
don't see that rewrite code anymore. Just checking - it is obsolete/deleted?
The graph rewrite is still performed before:
https://github.com/apache/beam/pull/7748/files#diff-2133745bace88d112aee725d114c2940R85
I don't think the rewrite could change the staged files. They are
effectively the classpath which won't change during pipeline translation.
Btw, I reviewed the pull request (https://github.com/apache/beam/pull/4760)
and also did not spot it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 195261)
Time Spent: 40m (was: 0.5h)
> Flink Runner prepares to-be-staged file too late
> -------------------------------------------------
>
> Key: BEAM-6608
> URL: https://issues.apache.org/jira/browse/BEAM-6608
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Affects Versions: 2.10.0
> Reporter: Maximilian Michels
> Assignee: Maximilian Michels
> Priority: Major
> Fix For: 2.11.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Before execution, the Flink Runner prepares all necessary files, in order for
> them to be shipped to the remote cluster.
> Due to a recent regression via BEAM-2873, the preparation is performed after
> the execution environment has been initialized. This skips the preparation
> phase and results in invalid files being shipped.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
