[
https://issues.apache.org/jira/browse/HDDS-284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16702298#comment-16702298
]
Hudson commented on HDDS-284:
-----------------------------
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15518 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/15518/])
HDDS-284. Checksum for ChunksData. (hanishakoneru: rev
64a4b6b08baf68c47cb17079bad2190f86166f50)
* (add)
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/Checksum.java
* (edit)
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/ChunkInputStream.java
* (edit)
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestChunkManagerImpl.java
* (edit)
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/TestBlockDeletingService.java
* (edit)
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/ChunkOutputStream.java
* (edit)
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ChunkInfo.java
* (add)
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/ChecksumData.java
* (edit) hadoop-hdds/common/src/main/proto/DatanodeContainerProtocol.proto
* (edit)
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/ChunkManagerImpl.java
* (edit)
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/io/ChunkGroupOutputStream.java
* (edit)
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/ContainerTestHelper.java
* (add)
hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/common/TestChecksum.java
* (edit)
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestOzoneRpcClient.java
* (add)
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/OzoneChecksumException.java
* (edit)
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java
* (edit)
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/helpers/ChunkUtils.java
* (edit)
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java
* (edit)
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/impl/TestContainerPersistence.java
* (edit)
hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/storage/DistributedStorageHandler.java
> CRC for ChunksData
> ------------------
>
> Key: HDDS-284
> URL: https://issues.apache.org/jira/browse/HDDS-284
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Reporter: Bharat Viswanadham
> Assignee: Hanisha Koneru
> Priority: Major
> Fix For: 0.4.0
>
> Attachments: CRC and Error Detection for Containers.pdf,
> HDDS-284.00.patch, HDDS-284.005.patch, HDDS-284.006.patch, HDDS-284.01.patch,
> HDDS-284.02.patch, HDDS-284.03.patch, HDDS-284.04.patch, Interleaving CRC and
> Error Detection for Containers.pdf
>
>
> This Jira is to add CRC for chunks data.
> Right now a Chunk Info structure looks like this:
> _message ChunkInfo {_
> _required string chunkName =_ _1__;_
> _required uint64 offset =_ _2__;_
> _required uint64 len =_ _3__;_
> _optional string checksum =_ _4__;_
> _repeated KeyValue metadata =_ _5__;_
> _}_
>
> Proposal is to change ChunkInfo structure as below:
> _message ChunkInfo {_
> _required string chunkName = 1 ;_
> _required uint64 offset = 2 ;_
> _required uint64 len = 3 ;_
> _repeated KeyValue metadata = 4;_
> _required ChecksumData checksumData = 5;_
> _}_
>
> The ChecksumData structure would be as follows:
> _message ChecksumData {_
> _required ChecksumType type = 1;_
> _required uint32 bytesPerChecksum = 2;_
> _repeated bytes checksums = 3;_
> _}_
>
> Instead of changing disk format, we put the checksum into chunkInfo.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
