[
https://issues.apache.org/jira/browse/FLINK-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318333#comment-15318333
]
Stephan Ewen commented on FLINK-3777:
-------------------------------------
I understand that these allow initialization across input splits. My feeling is
that if you need to initialize something a million times per input format, you
are probably not creating the splits correctly.
You will still have overhead for the communication of the splits, complexity
and potential memory issues on the JobManager for millions of splits.
The right fix for that would be to create the splits more correctly, rather
than add workaround methods to make the per-split work in the input format
less, at the cost of making the API harder to maintain.
> Add open and close methods to manage IF lifecycle
> -------------------------------------------------
>
> Key: FLINK-3777
> URL: https://issues.apache.org/jira/browse/FLINK-3777
> Project: Flink
> Issue Type: Improvement
> Components: Core
> Affects Versions: 1.0.1
> Reporter: Flavio Pompermaier
> Assignee: Flavio Pompermaier
> Labels: inputformat, lifecycle
>
> At the moment the opening and closing of an inputFormat are not managed,
> although open() could be (improperly IMHO) simulated by configure().
> This limits the possibility to reuse expensive resources (like database
> connections) and manage their release.
> Probably the best option would be to add 2 methods (i.e. openInputformat()
> and closeInputFormat() ) to RichInputFormat*
> * NOTE: the best option from a "semantic" point of view would be to rename
> the current open() and close() to openSplit() and closeSplit() respectively
> while using open() and close() methods for the IF lifecycle management, but
> this would cause a backward compatibility issue...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
