[
https://issues.apache.org/jira/browse/HBASE-13588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14525638#comment-14525638
]
Ted Yu commented on HBASE-13588:
--------------------------------
TestExportSnapshot#testExportRetry fails with patch locally.
In test output, I saw:
{code}
2015-05-02 21:06:37,106 WARN [ContainersLauncher #0]
nodemanager.DefaultContainerExecutor(207): Exit code from container
container_1430625969815_0001_01_000003 is : 143
Error: java.io.IOException: checksums don't match for input:
hdfs://localhost:60254/user/tyu/test-data/98603dee-5efc-43d1-a51e-cd70354870a9/data/default/testtb-1430625973413/
5373881af77da1f7f8463f160a317c15/cf/381595c4c327402fab3f3780b59f121b and
output:
file:/Users/tyu/1-hbase/hbase-server/target/test-data/7a27b9b5-48e4-4b27-b29d-16ecd441778e/local-export-
1430625977202/archive/data/default/testtb-1430625973413/5373881af77da1f7f8463f160a317c15/cf/381595c4c327402fab3f3780b59f121b
at
org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.copyFile(ExportSnapshot.java:299)
at
org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.map(ExportSnapshot.java:193)
at
org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.map(ExportSnapshot.java:119)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
{code}
> ExportSnapshot should checksum data transferred
> -----------------------------------------------
>
> Key: HBASE-13588
> URL: https://issues.apache.org/jira/browse/HBASE-13588
> Project: HBase
> Issue Type: Bug
> Affects Versions: 2.0.0, 1.0.1, 0.98.13
> Reporter: Mubeen Afzal
> Assignee: Mubeen Afzal
> Attachments: HBASE-13588.patch
>
>
> We sent quite a bit of data using ExportSnapshot and noticed some corruption.
> We use checksums to verify if we have transferred a particular file but
> never do any verification after the file has been transferred. I propose
> that we do a checksum upon completing file transfer and if the checksums do
> not match up, I propose we fail by throwing an IOException.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
