Oh, cool! Thanks Anoop!
2013/2/26 Anoop Sam John <[email protected]>: > JM > Pls check "dfs.datanode.scan.period.hours" > > -Anoop- > ________________________________________ > From: Jean-Marc Spaggiari [[email protected]] > Sent: Tuesday, February 26, 2013 7:04 PM > To: [email protected] > Subject: Re: HBase CheckSum vs Hadoop CheckSum > > Thanks for your replies. Few seconds I was feeling unsecured ;) > > Seems the default period for the DataBlockScanner is 3 weeks: > static final long DEFAULT_SCAN_PERIOD_HOURS = 21*24L; > > And I have not found anyway to modify that. I will continue to search > and might drop a msg on hadoop list if I still don't find. > > Thanks, > > JM > > 2013/2/26 Anoop Sam John <[email protected]>: >> I was typing a reply and by the time Liang replied :) >> Ya agree with him. It is only the HDFS client (At RS) not doing the >> checksum verification based on the HDFS stored checksum. >> Instead HBase only check for the correctness by comparing with stored >> checksum values. Still the periodic operation of block scanning at HDFS will >> continue. We can turn this OFF by configuring this period with a -ve value I >> think. >> >> -Anoop- >> ________________________________________ >> From: 谢良 [[email protected]] >> Sent: Tuesday, February 26, 2013 5:54 PM >> To: [email protected] >> Subject: 答复: HBase CheckSum vs Hadoop CheckSum >> >> comments in line >> >> Regards, >> Liang >> ________________________________________ >> 发件人: Jean-Marc Spaggiari [[email protected]] >> 发送时间: 2013年2月26日 20:03 >> 收件人: user >> 主题: HBase CheckSum vs Hadoop CheckSum >> >> Hi, >> >> Quick question. >> >> When we are activating the short circuit read in HBase, it's >> recommanded to activate the HBase checksum instead of Hadoop ones. >> This is done in the HBase configuration. >> >> I'm wondering what is the impact on the DataNode Block Scanner. >> >> Is it going to be stopped because checksums can't be used anymore? Or >> will Hadoop continue to store its own checksum and use them but it's >> just that HBase will not look at them anymore and will store and use >> its own checksums? >> [liang xie]: yes, still store checksum in meta file in current community >> version. >> btw, facebook's hadoop-fb20 branch has an inline checksum feature,IIRC >> >> Since it's an HBase configuration (hbase.regionserver.checksum.verify) >> I'm expecting this to not have any impact on the Block Scanner, but >> I'm looking for a confirmation. >> [liang xie]: yes, no impact on hdfs's DataBlockScanner, you can check >> detail in datanode's BlockPoolSliceScanner.verifyBlock(): >> blockSender = new BlockSender(block, 0, -1, false, true, true, >> datanode, null); >> >> >> Thanks, >> >> JM
