[
https://issues.apache.org/jira/browse/FLINK-4037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331982#comment-15331982
]
Stefan Richter edited comment on FLINK-4037 at 6/15/16 4:14 PM:
----------------------------------------------------------------
I think introducing an {{ArchivedExecutionGraph}} to maintain relevant
information from the {{ExecutionGraph}} of finished jobs is a little more
involved. E.g. the web interface has to deal with objects from
{{ArchivedExecutionGraph}} as well as {{ExecutionGraph}} to display finished
and in-flight jobs.
To achieve this, we could for example:
* Make {{ExecutionGraph}} a subclass of {{ArchivedExecutionGraph}}. However
this probably not the best idea as it violates the substitution principle.
* Introduce a common interface for both, {{ExecutionGraph}} and
{{ArchivedExecutionGraph}}. This interface provides means for the web interface
to extract information for display.
The method {{prepareForArchiving()}} could convert ExecutionGraphs into
ArchivedExecutionGraphs. We should ensure that asynchronous request by the web
interface are routed to a valid ExecutionGraph until the conversion is complete
and the object that acts as model for the web interface is substituted. Maybe
there is some cleaner way of introducing a real life cycle for ExecutionGraphs?
Furthermore, we need to identify all references in ExecutionGraph that could
hold objects from user-provided classes (e.g. accumulators, metrics, ...),
stringify their information (see {{JobConfigHandler.handleRequest()}}), and
release the references so that the user classloader can be garbage-collected.
Corresponding parts of the mentioned stringification have to be pushed into the
ExecutionGraph implementations. The web interface needs to be changed to
extract the information through the interface.
was (Author: srichter):
I think introducing an {{ArchivedExecutionGraph}} to maintain relevant
information from the {{ExecutionGraph}} of finished jobs is a little more
involved. E.g. the web interface has to deal with objects from
{{ArchivedExecutionGraph}} as well as {{ExecutionGraph}} to display finished
and in-flight jobs.
To achieve this, we could for example:
* Make {{ExecutionGraph}} a subclass of {{ArchivedExecutionGraph}}. However
this probably not the best idea as it violates the substitution principle.
* Introduce a common interface for both, {{ExecutionGraph}} and
{{ArchivedExecutionGraph}}. This interface provides means for the web interface
to extract information for display.
The method {{prepareForArchiving()}} could convert ExecutionGraphs into
ArchivedExecutionGraphs. We should ensure that asynchronous request by the web
interface are routed to a valid ExecutionGraph until the conversion is complete
and the object that acts as model for the web interface is substituted.
Furthermore, we need to identify all references in ExecutionGraph that could
hold objects from user-provided classes (e.g. accumulators, metrics, ...),
stringify their information (see {{JobConfigHandler.handleRequest()}}), and
release the references so that the user classloader can be garbage-collected.
Corresponding parts of the mentioned stringification have to be pushed into the
ExecutionGraph implementations. The web interface needs to be changed to
extract the information through the interface.
> Introduce ArchivedExecutionGraph without any user classes
> ---------------------------------------------------------
>
> Key: FLINK-4037
> URL: https://issues.apache.org/jira/browse/FLINK-4037
> Project: Flink
> Issue Type: Improvement
> Components: Webfrontend
> Reporter: Robert Metzger
> Assignee: Stefan Richter
>
> As a follow up to FLINK-4011: In order to allow the JobManager to unload all
> classes from a finished job, we need to convert the ExecutionGraph (and some
> attached objects like the ExecutionConfig) into a stringified version, not
> containing any user classes.
> The web frontend can show strings only anyways.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
