[
https://issues.apache.org/jira/browse/HBASE-10622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13917556#comment-13917556
]
Jerry He commented on HBASE-10622:
----------------------------------
A few more comments (since you are doing the improvement ...)
{code}
- // Verify that the written size match
- if (totalBytesWritten != inputFileSize) {
- String msg = "number of bytes copied not matching copied=" +
totalBytesWritten +
- " expected=" + inputFileSize + " for file=" + inputPath;
- throw new IOException(msg);
{code}
You think this is unnecessary?
In the run(), can we cleanup/delete snapshotTmpDir if Step 2 failed so that we
don't ask the user to manually clean it since it comes from our Step 1 copy?
Can we add a job counter say 'COPIES_FILES' to be along side with
'BYTES_COPIED'?
Another issue is probably more involved, and does not need to be covered in
this JIRA. It is the overall progress reporting of the ExportSnapshot job.
For example,
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot snapshot1
-copy-to /user/biadmin/mysnapshots -mappers 30
{code}
14/03/02 12:19:54 INFO mapred.JobClient: map 0% reduce 0%
14/03/02 12:20:12 INFO mapred.JobClient: map 6% reduce 0%
14/03/02 12:20:13 INFO mapred.JobClient: map 44% reduce 0%
14/03/02 12:20:19 INFO mapred.JobClient: map 83% reduce 0%
{code}
There is about 130G to export. But it takes just a few secs to get to 83%,
after the first around of mappers are launched, and will stay there for a long
time.
Similarly at the end it will show 100% for a long time while there are mappers
still running.
he map progress percentage is quite inaccurate with regard to the over
progress.
> Improve log and Exceptions in Export Snapshot
> ----------------------------------------------
>
> Key: HBASE-10622
> URL: https://issues.apache.org/jira/browse/HBASE-10622
> Project: HBase
> Issue Type: Bug
> Components: snapshots
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Fix For: 0.99.0
>
> Attachments: HBASE-10622-v0.patch, HBASE-10622-v1.patch,
> HBASE-10622-v2.patch, HBASE-10622-v3.patch
>
>
> from the logs of export snapshot is not really clear what's going on,
> adding some extra information useful to debug, and in some places the real
> exception can be thrown
--
This message was sent by Atlassian JIRA
(v6.2#6252)
