[
https://issues.apache.org/jira/browse/FLINK-2396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17323737#comment-17323737
]
Flink Jira Bot commented on FLINK-2396:
---------------------------------------
This issue is assigned but has not received an update in 7 days so it has been
labeled "stale-assigned". If you are still working on the issue, please give an
update and remove the label. If you are no longer working on the issue, please
unassign so someone else may work on it. In 7 days the issue will be
automatically unassigned.
> Review the datasets of dynamic path and static path in iteration.
> -----------------------------------------------------------------
>
> Key: FLINK-2396
> URL: https://issues.apache.org/jira/browse/FLINK-2396
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Task
> Reporter: Chengxiang Li
> Assignee: Chengxiang Li
> Priority: Major
> Labels: stale-assigned
>
> Currently Flink would cached dataset in static path as it assumes that
> dataset stay the same during the iteration, but this assumption does not
> always be true. Take sampling for example, the iteration data set is
> something like the weight vector of model and there is another training
> dataset from which to take a small sample to update the weight vector in each
> iteration (e.g. Stochastic Gradient Descent), we expect sampled dataset is
> different in each iteration, but Flink would cache the sampled dataset as it
> in static path.
> We should review how Flink identify dynamic path and static path, and support
> add sampled dataset in above example to dynamic path.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
