[
https://issues.apache.org/jira/browse/HDFS-9040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Walter Su updated HDFS-9040:
----------------------------
Attachment: HDFS-9040.00.patch
DFSStripedOutputStream will get exception only when BGDataStreamer fails, that
is to say, left < 6 streamers are healthy. BGDataStreamer handles exceptions
inside from StripedDataStreamer. I'll add this {{checkStreamers}} logic later.
a cellSize of 64k also works. I end a premature {{currentPacket}} to make it
aligned with cellSize boundary. A cellSize equals to packetSize is for reducing
packet number. Increasing size of body part of packet to 64k is another option.
The complicated part is, when a streamer#0 ends, you can't bump GS for it. So
before put an empty_last_packet to streamer#0, I'll check if there is a
recovery on-going.
I'm be glad if you could give some opinions first before I keep working on it
this weekend.
> Erasure coding: A BlockGroupDataStreamer to rule all internal blocks streamers
> ------------------------------------------------------------------------------
>
> Key: HDFS-9040
> URL: https://issues.apache.org/jira/browse/HDFS-9040
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Walter Su
> Assignee: Walter Su
> Attachments: HDFS-9040.00.patch
>
>
> A {{BlockGroupDataStreamer}} to communicate with NN to allocate/update block,
> and {{StripedDataStreamer}} s only have to stream blocks to DNs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
