[
https://issues.apache.org/jira/browse/APEXMALHAR-2484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15986306#comment-15986306
]
ASF GitHub Bot commented on APEXMALHAR-2484:
--------------------------------------------
GitHub user chaithu14 opened a pull request:
https://github.com/apache/apex-malhar/pull/614
APEXMALHAR-2484 Support of PartFileWriter for writing the part files
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/chaithu14/incubator-apex-malhar
APEXMALHAR-2484-partfilewriter
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/apex-malhar/pull/614.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #614
----
commit d45ea93d8db24af6181d55a5b4e2a13148e4629b
Author: chaitanya <[email protected]>
Date: 2017-04-27T10:00:09Z
APEXMALHAR-2484 Support of PartFileWriter for writing the part files
----
> BlockWriter for writing the part files into the specified directory
> -------------------------------------------------------------------
>
> Key: APEXMALHAR-2484
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2484
> Project: Apache Apex Malhar
> Issue Type: Task
> Reporter: Chaitanya
> Assignee: Chaitanya
>
> Use case: Suppose, the size of source file (f1.txt) is 1 GB and the block
> size is 128 MB. I want to copy the file in destination as follows:
> f1.txt.part1
> f2.txt.part2
> ....
> By default, size of each part file is 128 MB except the last part.
> Design: Currently, the BlockWriter is restricted to write the part files into
> the HDFS on which the app is running. To achieve the above use case, operator
> needs the block index and relative path information. BlockMetadata which is
> the input port for the BlockWriter doesn't have these information.
> So, I am creating the new operator(PartFileWriter) which extends from
> BlockWriter with the input port of type FileMetadata.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
