[
https://issues.apache.org/jira/browse/HAMA-258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145361#comment-13145361
]
Thomas Jungblut commented on HAMA-258:
--------------------------------------
Multiple splits per tasks should be actually easy, we can use the combine file
split:
http://hadoop.apache.org/mapreduce/docs/r0.21.0/api/org/apache/hadoop/mapreduce/lib/input/CombineFileSplit.html
It will take care of the equally distribution of the blocks. You can provide
via "mapred.max.split.size" a split size for a task.
We just have to calculate this split size. But it is trivial, sum(block_sizes)
/ number of tasks.
Or the user could directly set this size.
I'm not to sure currently.
> Design a input and output system
> --------------------------------
>
> Key: HAMA-258
> URL: https://issues.apache.org/jira/browse/HAMA-258
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Affects Versions: 0.3.0
> Reporter: Edward J. Yoon
> Assignee: Edward J. Yoon
> Fix For: 0.4.0
>
> Attachments: HAMA-258_improved.patch, IONoInput.patch, io_v01.patch,
> io_v02.patch, io_v03.patch, io_v04.patch
>
>
> This issue will handle the input and output system with data splitter.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
