[
https://issues.apache.org/jira/browse/APEXMALHAR-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15858851#comment-15858851
]
Sanjay M Pujare edited comment on APEXMALHAR-2403 at 2/9/17 1:58 AM:
---------------------------------------------------------------------
That is (or can) not be done here. The user has decided to shut down the
application (thru Apex CLI) because he has noticed that the app has finished
reading the whole SQL table and has determined that no more records are going
to be added. Shouldn't the shutdown command tell the FileOutoutOperator to
finalize the files i.e. rename from temp to actual names? This is an example of
a batch application where the completion of batch is determined externally (not
thru EOF on the input) and it will be good to support such use cases.
was (Author: sanjaypujare):
That is or cannot be done here. The user has decided to shut down the
application (thru Apex CLI) because he has noticed that the app has finished
reading the whole SQL table and has determined that no more records are going
to be added. Shouldn't the shutdown command tell the FileOutoutOperator to
finalize the files i.e. rename from temp to actual names? This is an example of
a batch application where the completion of batch is determined externally (not
thru EOF on the input) and it will be good to support such use cases.
> Finalization is not invoked for AbstractFileOutputOperator<INPUT>
> -----------------------------------------------------------------
>
> Key: APEXMALHAR-2403
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2403
> Project: Apache Apex Malhar
> Issue Type: Bug
> Reporter: Sanjay M Pujare
> Priority: Minor
>
> I used an example/template program for Database to HDFS data copy. While
> copying the table, I see "temp" files created on the output HDFS side which I
> expected to be "finalized" to non-temp names when the app is shut down. But
> that did not happen. AbstractFileOutputOperator<INPUT> and its subclasses
> should do proper finalization via the deactivate() method when an app is
> properly shut down.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
