[
https://issues.apache.org/jira/browse/HDDS-284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bharat Viswanadham updated HDDS-284:
------------------------------------
Description:
This Jira is to add CRC for chunks data.
Right now a Chunk Info structure looks like this:
_message ChunkInfo {_
_required string chunkName =_ _1__;_
_required uint64 offset =_ _2__;_
_required uint64 len =_ _3__;_
_optional string checksum =_ _4__;_
_repeated KeyValue metadata =_ _5__;_
_}_
_Proposal is to change ChunkInfo structure as below:_
_message ChunkInfo {_
_required string chunkName =_ _1__;_
_required uint64 offset =_ _2__;_
_required uint64 len =_ _3__;_
_optional bytes checksum =_ _4__;_
_optional CRCType checksumType =_ _5__;_
_optional string legacyMetadata =_ _6__;_
_optional string legacyData =_ _7__;_
_repeated KeyValue metadata =_ _8__;_
_}_
_Instead of changing disk format, we put the checksum, checksumtype and legacy
data fields in to chunkInfo._
was:
This Jira is to add CRC for chunks data.
Right now, in chunkInfo, data is just byte array. We want to change this as
below:
_message Data {_
_required string magic =”Tullys00” = 1;_
_required CRCTYPE crcType =_ _2__;_
_optional String LegacyMetadata = 3; // Fields to support inplace data
migration_
_optional String LegacyData = 4;_
_optional ushort ChecksumblockSize = 5; // Size of the block used to compute
the checksums_
_repeated uint32 checksums = 6 ; // Set of Checksums_
*_repeated byte data = 6;_* _// Actual Data stream_
_}_
_This will help in error detection for containers during container scanner._
> Interleaving CRC for ChunksData
> -------------------------------
>
> Key: HDDS-284
> URL: https://issues.apache.org/jira/browse/HDDS-284
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Reporter: Bharat Viswanadham
> Assignee: Bharat Viswanadham
> Priority: Major
> Fix For: 0.3.0
>
> Attachments: HDDS-284.00.patch, HDDS-284.01.patch, HDDS-284.02.patch,
> HDDS-284.03.patch, Interleaving CRC and Error Detection for Containers.pdf
>
>
> This Jira is to add CRC for chunks data.
>
>
> Right now a Chunk Info structure looks like this:
>
> _message ChunkInfo {_
> _required string chunkName =_ _1__;_
> _required uint64 offset =_ _2__;_
> _required uint64 len =_ _3__;_
> _optional string checksum =_ _4__;_
> _repeated KeyValue metadata =_ _5__;_
> _}_
>
> _Proposal is to change ChunkInfo structure as below:_
>
> _message ChunkInfo {_
> _required string chunkName =_ _1__;_
> _required uint64 offset =_ _2__;_
> _required uint64 len =_ _3__;_
> _optional bytes checksum =_ _4__;_
> _optional CRCType checksumType =_ _5__;_
> _optional string legacyMetadata =_ _6__;_
> _optional string legacyData =_ _7__;_
> _repeated KeyValue metadata =_ _8__;_
> _}_
>
> _Instead of changing disk format, we put the checksum, checksumtype and
> legacy data fields in to chunkInfo._
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
