[
https://issues.apache.org/jira/browse/HDFS-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ewan Higgs updated HDFS-11639:
------------------------------
Attachment: HDFS-11639-HDFS-9806.003.patch
Attaching an updated patch that addresses some of [~virajith]'s comments:
{quote}
- In ProvidedBlocksBuilder#newLocatedBlock, the fileRegion should be
resolved only if the block has PROVIDED locations (i.e.,
hasProvidedLocations
is true). When dfs.namenode.provided.enabled is set to true, all
LocatedBlock
are created in this method, and for non-provided blocks, a resolution of
BlockAlias is needed.
- PBHelperClient#convertLocatedBlockProto() and
PBHelperClient#convertLocatedBlock() should be modified to decode/encode
the
BlockAlias bytes.
- How about decoding the blockAlias bytes in DataXceiver#readBlock using
a new DataTransferProtoUtil#blockAliasFromProto(bytes[] blockAlias method
instead of using the BlockAlias#builder()? The former will be in-line with
the way the protobufs are decoded in DataXceiver. Further, if in the
future a different BlockAlias is used, the current implementation of using
the FileRegion#Builder in DataXceiver#readBlock will be hard to extend
(will end up being try FileRegion#Builder, if null try
BlockAliasXX#Builder and so on).
- Similar to passing on BlockAlias from DataXceiver#readBlock to
BlockSender, it should be passed along from DataXceiver#readBlock to
BlockReceiver. However, we would not need it till we have writes
implemented.
- DataStreamer#blockAlias will never be non-null. I think it should
be initialized in DFSOutputStream.
{quote}
This also added BlockAlias to the transferBlocks message but it isn't
incorporated in since this is a write message. e.g. {{DNA_TRANSFER}} has not
yet been updated.
> [READ] Encode the BlockAlias in the client protocol
> ---------------------------------------------------
>
> Key: HDFS-11639
> URL: https://issues.apache.org/jira/browse/HDFS-11639
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs
> Reporter: Ewan Higgs
> Assignee: Ewan Higgs
> Attachments: HDFS-11639-HDFS-9806.001.patch,
> HDFS-11639-HDFS-9806.002.patch, HDFS-11639-HDFS-9806.003.patch
>
>
> As part of the {{PROVIDED}} storage type, we have a {{BlockAlias}} type which
> encodes information about where the data comes from. i.e. URI, offset,
> length, and nonce value. This data should be encoded in the protocol
> ({{LocatedBlockProto}} and the {{BlockTokenIdentifier}}) when a block is
> available using the PROVIDED storage type.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
