[jira] [Resolved] (HBASE-5743) Support GIT patches
[ https://issues.apache.org/jira/browse/HBASE-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg resolved HBASE-5743. Resolution: Fixed Fix Version/s: 0.89-fb 0.96.0 0.94.0 Support GIT patches --- Key: HBASE-5743 URL: https://issues.apache.org/jira/browse/HBASE-5743 Project: HBase Issue Type: Bug Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Fix For: 0.94.0, 0.96.0, 0.89-fb Attachments: HBASE-5743-2.patch, HBASE-5743.patch Need to import HADOOP-7384 into our version of Hadoop QA to allow test-patch to be more flexible about patch format -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-3976) Disable Block Cache On Compactions
[ https://issues.apache.org/jira/browse/HBASE-3976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg resolved HBASE-3976. Resolution: Incomplete its very hard to tell if this issue is corrected in trunk or not. Talked offline with Mikhail about strategies for getting proper resolution here. HBASE-5230 has been made to verify that this issue is indeed fixed will encompass a fix if other subtle bugs remain. Disable Block Cache On Compactions -- Key: HBASE-3976 URL: https://issues.apache.org/jira/browse/HBASE-3976 Project: HBase Issue Type: Improvement Components: regionserver Affects Versions: 0.90.3 Reporter: Karthick Sankarachary Assignee: Nicolas Spiegelberg Priority: Minor Attachments: HBASE-3976-V3.patch, HBASE-3976-unconditional.patch, HBASE-3976.patch Is there a good reason to believe that caching blocks during compactions is beneficial? Currently, if block cache is enabled on a certain family, then every time it's compacted, we load all of its blocks into the (LRU) cache, at the expense of the legitimately hot ones. As a matter of fact, this concern was raised earlier in HBASE-1597, which rightly points out that, we should not bog down the LRU with unneccessary blocks during compaction. Even though that issue has been marked as fixed, it looks like it ought to be reopened. Should we err on the side of caution and not cache blocks during compactions period (as illustrated in the attached patch)? Or, can we be selectively aggressive about what blocks do get cached during compaction (e.g., only cache those blocks from the recent files)? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4770) Enable per column family compaction for a region via hbase shell
[ https://issues.apache.org/jira/browse/HBASE-4770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg resolved HBASE-4770. Resolution: Duplicate Duplicate of HBASE-4913 Enable per column family compaction for a region via hbase shell Key: HBASE-4770 URL: https://issues.apache.org/jira/browse/HBASE-4770 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0 Reporter: Nicolas Spiegelberg Assignee: Pritam Damania Priority: Minor Pretty self-explanatory. We have per-table per-region compaction. However, there are use cases where you have 10+ column families. It is useful to be able to specify the actual CF you wish to compact. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5126) AND (USING FilterList) of two ColumnPrefixFilters broken
[
https://issues.apache.org/jira/browse/HBASE-5126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicolas Spiegelberg resolved HBASE-5126.
Resolution: Not A Problem
not a problem in trunk, only 89-fb
AND (USING FilterList) of two ColumnPrefixFilters broken
Key: HBASE-5126
URL: https://issues.apache.org/jira/browse/HBASE-5126
Project: HBase
Issue Type: Bug
Reporter: Kannan Muthukkaruppan
Assignee: Madhuwanti Vaidya
Attachments: testAndTwoPrefixFilters.rb, testAndTwoPrefixFilters.rb
[Notice this in 89 branch. Possibly an issue in trunk also.]
A test which does a columnPrefixFilter(tag0) AND columnPrefixFilter(tag1)
should return 0 kvs; instead it returns kvs with prefix tag0.
{code}
table = HTable.new(conf, tableName)
put = Put.new(Bytes.toBytes(row))
put.add(cf1name, Bytes.toBytes(tag0), Bytes.toBytes(value0))
put.add(cf1name, Bytes.toBytes(tag1), Bytes.toBytes(value1))
put.add(cf1name, Bytes.toBytes(tag2), Bytes.toBytes(value2))
table.put(put)
# Test for AND Two Column Prefix Filters
filter1 = ColumnPrefixFilter.new(Bytes.toBytes(tag0));
filter2 = ColumnPrefixFilter.new(Bytes.toBytes(tag2));
filters = FilterList.new(FilterList::Operator::MUST_PASS_ALL);
filters.addFilter(filter1);
filters.addFilter(filter2);
get = Get.new(Bytes.toBytes(row))
get.setFilter(filters)
get.setMaxVersions();
keyValues = table.get(get).raw()
keyValues.each do |keyValue|
puts Key=#{Bytes.toStringBinary(keyValue.getQualifier())};
Value=#{Bytes.toStringBinary(keyValue.getValue())};
Timestamp=#{keyValue.getTimestamp()}
end
{code}
outputs:
{code}
Key=tag0; Value=value0; Timestamp=1325719223523
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5031) [89-fb] Remove hard-coded non-existent host name from TestScanner
[ https://issues.apache.org/jira/browse/HBASE-5031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg resolved HBASE-5031. Resolution: Fixed Release Note: Fixed in 0.89-fb. Not necessary in trunk [89-fb] Remove hard-coded non-existent host name from TestScanner -- Key: HBASE-5031 URL: https://issues.apache.org/jira/browse/HBASE-5031 Project: HBase Issue Type: Bug Reporter: Mikhail Bautin Priority: Minor Attachments: D867.1.patch TestScanner is failing on 0.89-fb because it has a hard-coded fake host name that it is trying to look up. Replacing this with 127.0.0.1:random_port instead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5021) Enforce upper bound on timestamp
[ https://issues.apache.org/jira/browse/HBASE-5021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg resolved HBASE-5021. Resolution: Fixed stack fix the TestHeapSize problem in a subsequent commit. This JIRA shouldn't have been reopened. Enforce upper bound on timestamp Key: HBASE-5021 URL: https://issues.apache.org/jira/browse/HBASE-5021 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Critical Fix For: 0.94.0 Attachments: 5021-addendum.txt, D849.1.patch, D849.2.patch, D849.3.patch, HBASE-5021-trunk.patch We have been getting hit with performance problems on our time-series database due to invalid timestamps being inserted by the timestamp. We are working on adding proper checks to app server, but production performance could be severely impacted with significant recovery time if something slips past. Since timestamps are considered a fundamental part of the HBase schema multiple optimizations use timestamp information, we should allow the option to sanity check the upper bound on the server-side in HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4440) add an option to presplit table to PerformanceEvaluation
[ https://issues.apache.org/jira/browse/HBASE-4440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg resolved HBASE-4440. Resolution: Fixed Fix Version/s: 0.94.0 thanks Sujee! add an option to presplit table to PerformanceEvaluation Key: HBASE-4440 URL: https://issues.apache.org/jira/browse/HBASE-4440 Project: HBase Issue Type: Improvement Components: util Reporter: Sujee Maniyam Priority: Minor Labels: benchmark Fix For: 0.94.0 Attachments: PerformanceEvaluation.java, PerformanceEvaluation_HBASE_4440.patch, PerformanceEvaluation_HBASE_4440_2.patch PerformanceEvaluation a quick way to 'benchmark' a HBase cluster. The current 'write*' operations do not pre-split the table. Pre splitting the table will really boost the insert performance. It would be nice to have an option to enable pre-splitting table before the inserts begin. it would look something like: (a) hbase ...PerformanceEvaluation --presplit=10 other options (b) hbase ...PerformanceEvaluation --presplit other options (b) will try to presplit the table on some default value (say number of region servers) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4911) Clean shutdown
[ https://issues.apache.org/jira/browse/HBASE-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg resolved HBASE-4911. Resolution: Fixed Talked with Mikhail more about this issue. It is fixed in trunk currently Clean shutdown -- Key: HBASE-4911 URL: https://issues.apache.org/jira/browse/HBASE-4911 Project: HBase Issue Type: Sub-task Components: client, regionserver Reporter: Nicolas Spiegelberg Fix For: 0.94.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4787) Make corePool as a configurable parameter in HTable
[ https://issues.apache.org/jira/browse/HBASE-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg resolved HBASE-4787. Resolution: Fixed Fix Version/s: 0.94.0 Make corePool as a configurable parameter in HTable --- Key: HBASE-4787 URL: https://issues.apache.org/jira/browse/HBASE-4787 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Priority: Trivial Fix For: 0.94.0 Attachments: HBASE-4787.patch Make the corePool a configurable parameter in HTable. So we can tune this parameter in the config file. While at it, change the core pool name so we can distinguish it from other AppServer pools. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-2856) TestAcidGuarantee broken on trunk
[ https://issues.apache.org/jira/browse/HBASE-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg resolved HBASE-2856. Resolution: Fixed TestAcidGuarantee broken on trunk -- Key: HBASE-2856 URL: https://issues.apache.org/jira/browse/HBASE-2856 Project: HBase Issue Type: Bug Affects Versions: 0.89.20100621 Reporter: ryan rawson Assignee: Amitanand Aiyer Priority: Blocker Fix For: 0.94.0 Attachments: 2856-v2.txt, 2856-v3.txt, 2856-v4.txt, 2856-v5.txt, 2856-v6.txt, 2856-v7.txt, 2856-v8.txt, 2856-v9-all-inclusive.txt, acid.txt TestAcidGuarantee has a test whereby it attempts to read a number of columns from a row, and every so often the first column of N is different, when it should be the same. This is a bug deep inside the scanner whereby the first peek() of a row is done at time T then the rest of the read is done at T+1 after a flush, thus the memstoreTS data is lost, and previously 'uncommitted' data becomes committed and flushed to disk. One possible solution is to introduce the memstoreTS (or similarly equivalent value) to the HFile thus allowing us to preserve read consistency past flushes. Another solution involves fixing the scanners so that peek() is not destructive (and thus might return different things at different times alas). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4485) Eliminate window of missing Data
[
https://issues.apache.org/jira/browse/HBASE-4485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicolas Spiegelberg resolved HBASE-4485.
Resolution: Fixed
Eliminate window of missing Data
Key: HBASE-4485
URL: https://issues.apache.org/jira/browse/HBASE-4485
Project: HBase
Issue Type: Sub-task
Reporter: Amitanand Aiyer
Assignee: Amitanand Aiyer
Fix For: 0.94.0
Attachments: 4485-v1.diff, 4485-v2.diff, 4485-v3.diff, 4485-v4.diff,
4485-v5.diff, 4485-v6.diff, repro_bug-4485.diff
After incorporating v11 of the 2856 fix, we discovered that we are still
having some ACID violations.
This time, however, the problem is not about including newer updates; but,
about missing older updates
that should be including.
Here is what seems to be happening.
There is a race condition in the StoreScanner.getScanners()
private ListKeyValueScanner getScanners(Scan scan,
final NavigableSetbyte[] columns) throws IOException {
// First the store file scanners
ListStoreFileScanner sfScanners = StoreFileScanner
.getScannersForStoreFiles(store.getStorefiles(), cacheBlocks,
isGet, false);
ListKeyValueScanner scanners =
new ArrayListKeyValueScanner(sfScanners.size()+1);
// include only those scan files which pass all filters
for (StoreFileScanner sfs : sfScanners) {
if (sfs.shouldSeek(scan, columns)) {
scanners.add(sfs);
}
}
// Then the memstore scanners
if (this.store.memstore.shouldSeek(scan)) {
scanners.addAll(this.store.memstore.getScanners());
}
return scanners;
}
If for example there is a call to Store.updateStorefiles() that happens
between
the store.getStorefiles() and this.store.memstore.getScanners(); then
it is possible that there was a new HFile created, that is not seen by the
StoreScanner, and the data is not present in the Memstore.snapshot either.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4594) Ensure that KV's newer than the oldest-living-scanner is not accounted for the maxVersions during flush/compaction.
[ https://issues.apache.org/jira/browse/HBASE-4594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg resolved HBASE-4594. Resolution: Fixed Ensure that KV's newer than the oldest-living-scanner is not accounted for the maxVersions during flush/compaction. --- Key: HBASE-4594 URL: https://issues.apache.org/jira/browse/HBASE-4594 Project: HBase Issue Type: Sub-task Reporter: Amitanand Aiyer Priority: Minor Fix For: 0.94.0 Attachments: 4594-v1.diff, 4594-v2.diff -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4760) Add Developer Debug Options to HBase Config
[ https://issues.apache.org/jira/browse/HBASE-4760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg resolved HBASE-4760. Resolution: Duplicate Fix Version/s: 0.94.0 Duplicate of HBASE-4760 Add Developer Debug Options to HBase Config --- Key: HBASE-4760 URL: https://issues.apache.org/jira/browse/HBASE-4760 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0 Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Fix For: 0.94.0 Add in optional HBase configuration options that core developers will commonly use: an option to enable JDWP debugging an option to use a separate logfile for GC information. (Part of the effort to move 89-fb features over to trunk) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4674) splitLog silently fails
[ https://issues.apache.org/jira/browse/HBASE-4674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg resolved HBASE-4674. Resolution: Duplicate Fix Version/s: 0.92.0 Fixed by HBASE-2312 splitLog silently fails --- Key: HBASE-4674 URL: https://issues.apache.org/jira/browse/HBASE-4674 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Environment: splitLog() can fail silently and region can open w/o its edits getting replayed. Reporter: Prakash Khemani Assignee: Prakash Khemani Priority: Blocker Fix For: 0.92.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-2312) Possible data loss when RS goes into GC pause while rolling HLog
[ https://issues.apache.org/jira/browse/HBASE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg resolved HBASE-2312. Resolution: Fixed Possible data loss when RS goes into GC pause while rolling HLog Key: HBASE-2312 URL: https://issues.apache.org/jira/browse/HBASE-2312 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.90.0 Reporter: Karthik Ranganathan Assignee: Nicolas Spiegelberg Priority: Critical Fix For: 0.92.0 Attachments: D99.1.patch, D99.2.patch, D99.3.patch There is a very corner case when bad things could happen(ie data loss): 1)RS #1 is going to roll its HLog - not yet created the new one, old one will get no more writes 2)RS #1 enters GC Pause of Death 3)Master lists HLog files of RS#1 that is has to split as RS#1 is dead, starts splitting 4)RS #1 wakes up, created the new HLog (previous one was rolled) and appends an edit - which is lost The following seems like a possible solution: 1)Master detects RS#1 is dead 2)The master renames the /hbase/.logs/regionserver name directory to something else (say /hbase/.logs/regionserver name-dead) 3)Add mkdir support (as opposed to mkdirs) to HDFS - so that a file create fails if the directory doesn't exist. Dhruba tells me this is very doable. 4)RS#1 comes back up and is not able create the new hlog. It restarts itself. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4611) Add support for Phabricator/Differential as an alternative code review tool
[ https://issues.apache.org/jira/browse/HBASE-4611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg resolved HBASE-4611. Resolution: Fixed Fix Version/s: 0.94.0 0.92.0 we had the entire FB HBase team go through the steps I mentioned for using arc today. They were able to successfully install arc and use it to create an example diff (hence all the activity on this diff today). Any interested parties, please give it a whirl and give feedback. Add support for Phabricator/Differential as an alternative code review tool --- Key: HBASE-4611 URL: https://issues.apache.org/jira/browse/HBASE-4611 Project: HBase Issue Type: Task Reporter: Jonathan Gray Assignee: Nicolas Spiegelberg Fix For: 0.92.0, 0.94.0 Attachments: D153.1.patch, D165.1.patch, D165.2.patch, D171.1.patch, D177.1.patch, D177.2.patch, D183.1.patch, D189.1.patch, D201.1.patch, D207.1.patch, D21.1.patch, D21.1.patch From http://phabricator.org/ : Phabricator is a open source collection of web applications which make it easier to write, review, and share source code. It is currently available as an early release. Phabricator was developed at Facebook. It's open source so pretty much anyone could host an instance of this software. To begin with, there will be a public-facing instance located at http://reviews.facebook.net (sponsored by Facebook and hosted by the OSUOSL http://osuosl.org). We will use this JIRA to deal with adding (and ensuring) Apache-friendly support that will allow us to do code reviews with Phabricator for HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4591) TTL for old HLogs should be calculated from last modification time.
[ https://issues.apache.org/jira/browse/HBASE-4591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg resolved HBASE-4591. Resolution: Fixed Fix Version/s: 0.94.0 0.92.0 TTL for old HLogs should be calculated from last modification time. --- Key: HBASE-4591 URL: https://issues.apache.org/jira/browse/HBASE-4591 Project: HBase Issue Type: Improvement Components: master Affects Versions: 0.89.20100621 Reporter: Madhuwanti Vaidya Assignee: Madhuwanti Vaidya Priority: Minor Fix For: 0.92.0, 0.94.0 Attachments: 0007-HBASE-4591-Increase-TTL-for-old-HLogs-and-change-TTL.patch, HBASE-4591-2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4418) Show all the hbase configuration in the web ui
[ https://issues.apache.org/jira/browse/HBASE-4418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg resolved HBASE-4418. Resolution: Fixed Fix Version/s: 0.94.0 Show all the hbase configuration in the web ui -- Key: HBASE-4418 URL: https://issues.apache.org/jira/browse/HBASE-4418 Project: HBase Issue Type: Improvement Reporter: Liyin Tang Assignee: Liyin Tang Fix For: 0.94.0 Attachments: hbase-4418-apache-trunk.patch The motivation is to show ALL the HBase configuration, which takes effect in the run time, in a global place. So we can easily know which configuration takes effect and what the value is. The configuration shows all the HBase and DFS configuration entry in the configuration file and also includes all the HBase default setting in the code, which is not the config file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4078) Silent Data Offlining During HDFS Flakiness
[ https://issues.apache.org/jira/browse/HBASE-4078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg resolved HBASE-4078. Resolution: Fixed Fix Version/s: 0.94.0 0.92.0 added to 89, 92, 94 Silent Data Offlining During HDFS Flakiness --- Key: HBASE-4078 URL: https://issues.apache.org/jira/browse/HBASE-4078 Project: HBase Issue Type: Bug Components: io, regionserver Affects Versions: 0.89.20100924, 0.90.3, 0.92.0 Reporter: Nicolas Spiegelberg Assignee: Pritam Damania Priority: Blocker Fix For: 0.92.0, 0.94.0 Attachments: 0001-Validate-store-files-after-compactions-flushes.patch, 0001-Validate-store-files.patch See HBASE-1436 . The bug fix for this JIRA is a temporary workaround for improperly moving partially-written files from TMP into the region directory when a FS error occurs. Unfortunately, the fix is to ignore all IO exceptions, which masks off-lining due to FS flakiness. We need to permanently fix the problem that created HBASE-1436 then at least have the option to not open a region during times of flakey FS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4568) Make zk dump jsp response more quickly
[ https://issues.apache.org/jira/browse/HBASE-4568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg resolved HBASE-4568. Resolution: Fixed Fix Version/s: 0.94.0 0.92.0 Hadoop Flags: Reviewed Make zk dump jsp response more quickly -- Key: HBASE-4568 URL: https://issues.apache.org/jira/browse/HBASE-4568 Project: HBase Issue Type: Improvement Reporter: Liyin Tang Assignee: Liyin Tang Fix For: 0.92.0, 0.94.0 Attachments: HBASE-4568.patch 1) For each zk dump, currently hbase will create a zk client instance every time. This is quite slow when any machines in the quorum is dead. Because it will connect to each machine in the zk quorum again. code HMaster master = (HMaster)getServletContext().getAttribute(HMaster.MASTER); Configuration conf = master.getConfiguration(); HBaseAdmin hbadmin = new HBaseAdmin(conf); HConnection connection = hbadmin.getConnection(); ZooKeeperWatcher watcher = connection.getZooKeeperWatcher(); /code So we can simplify this: code HMaster master = (HMaster)getServletContext().getAttribute(HMaster.MASTER); ZooKeeperWatcher watcher = master.getZooKeeperWatcher(); /code 2) Also when hbase call getServerStats() for each machine in the zk quorum, it hard coded the default time out as 1 min. It would be nice to make this configurable and set it to a low time out. When hbase tries to connect to each machine in the zk quorum, it will create the socket, and then set the socket time out, and read it with this time out. It means hbase will create a socket and connect to the zk server with 0 time out at first, which will take a long time. Because a timeout of zero is interpreted as an infinite timeout. The connection will then block until established or an error occurs. 3) The recoverable zookeeper should be real exponentially backoff when there is connection loss exception, which will give hbase much longer time window to recover from zk machine failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
