JM
Pls check "dfs.datanode.scan.period.hours"
-Anoop-
________________________________________
From: Jean-Marc Spaggiari [[email protected]]
Sent: Tuesday, February 26, 2013 7:04 PM
To: [email protected]
Subject: Re: HBase CheckSum vs Hadoop CheckSum
Thanks for your replies. Few seconds I was feeling unsecured ;)
Seems the default period for the DataBlockScanner is 3 weeks:
static final long DEFAULT_SCAN_PERIOD_HOURS = 21*24L;
And I have not found anyway to modify that. I will continue to search
and might drop a msg on hadoop list if I still don't find.
Thanks,
JM
2013/2/26 Anoop Sam John <[email protected]>:
> I was typing a reply and by the time Liang replied :)
> Ya agree with him. It is only the HDFS client (At RS) not doing the checksum
> verification based on the HDFS stored checksum.
> Instead HBase only check for the correctness by comparing with stored
> checksum values. Still the periodic operation of block scanning at HDFS will
> continue. We can turn this OFF by configuring this period with a -ve value I
> think.
>
> -Anoop-
> ________________________________________
> From: 谢良 [[email protected]]
> Sent: Tuesday, February 26, 2013 5:54 PM
> To: [email protected]
> Subject: 答复: HBase CheckSum vs Hadoop CheckSum
>
> comments in line
>
> Regards,
> Liang
> ________________________________________
> 发件人: Jean-Marc Spaggiari [[email protected]]
> 发送时间: 2013年2月26日 20:03
> 收件人: user
> 主题: HBase CheckSum vs Hadoop CheckSum
>
> Hi,
>
> Quick question.
>
> When we are activating the short circuit read in HBase, it's
> recommanded to activate the HBase checksum instead of Hadoop ones.
> This is done in the HBase configuration.
>
> I'm wondering what is the impact on the DataNode Block Scanner.
>
> Is it going to be stopped because checksums can't be used anymore? Or
> will Hadoop continue to store its own checksum and use them but it's
> just that HBase will not look at them anymore and will store and use
> its own checksums?
> [liang xie]: yes, still store checksum in meta file in current community
> version.
> btw, facebook's hadoop-fb20 branch has an inline checksum feature,IIRC
>
> Since it's an HBase configuration (hbase.regionserver.checksum.verify)
> I'm expecting this to not have any impact on the Block Scanner, but
> I'm looking for a confirmation.
> [liang xie]: yes, no impact on hdfs's DataBlockScanner, you can check
> detail in datanode's BlockPoolSliceScanner.verifyBlock():
> blockSender = new BlockSender(block, 0, -1, false, true, true,
> datanode, null);
>
>
> Thanks,
>
> JM
