[jira] [Commented] (HBASE-12776) SpliTransaction: Log number of files to be split
[ https://issues.apache.org/jira/browse/HBASE-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260896#comment-14260896 ] Hudson commented on HBASE-12776: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #729 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/729/]) HBASE-12776 SpliTransaction: Log number of files to be split. (larsh: rev 3c5645f2b035af464ea0cf6181b241e1a002a501) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java > SpliTransaction: Log number of files to be split > > > Key: HBASE-12776 > URL: https://issues.apache.org/jira/browse/HBASE-12776 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 2.0.0, 0.98.10, 1.0.1, 1.1.0, 0.94.27 > > Attachments: 12765.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12767) Fix a StoreFileScanner NPE in reverse scan flow
[
https://issues.apache.org/jira/browse/HBASE-12767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260897#comment-14260897
]
Hudson commented on HBASE-12767:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #729 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/729/])
HBASE-12767 Fix a StoreFileScanner NPE in reverse scan flow (xieliang: rev
3197a631dbdf8cee7ba8fe1ea4864ac55fde5a43)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java
> Fix a StoreFileScanner NPE in reverse scan flow
> ---
>
> Key: HBASE-12767
> URL: https://issues.apache.org/jira/browse/HBASE-12767
> Project: HBase
> Issue Type: Bug
> Components: regionserver, Scanners
>Affects Versions: 1.0.0, 2.0.0, 0.98.9
>Reporter: Liang Xie
>Assignee: Liang Xie
> Fix For: 1.0.0, 2.0.0, 0.98.10, 1.1.0
>
> Attachments: HBASE-12767.txt
>
>
> we observed a NPE in our internal 0.94 branch prd env:
> ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekToPreviousRow(StoreFileScanner.java:409)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.backwardSeek(StoreFileScanner.java:467)
> at
> org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.seekScanners(ReversedStoreScanner.java:78)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:154)
> at
> org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.(ReversedStoreScanner.java:49)
> at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2216)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:3984)
> at
> org.apache.hadoop.hbase.regionserver.ReversedRegionScannerImpl.(ReversedRegionScannerImpl.java:43)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1851)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1837)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1812)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2752)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2715)
> After tracing, it turned out a boundary bug at HalfStoreFileReader. If we
> have a reverse scan request at "bottom" part, once the start row >= the last
> row at bottom part, the NPE will be encourtered, since we will invoke
> "next()", and "atEnd = true" will be called in HalfStoreFileReader, then when
> we want to do "seekBefore" or other sth in StoreFileScanner,
> StoreFileScanner.cur will call:
> {code}
> public Cell getKeyValue() {
> if (atEnd) return null;
> {code}
> The fix is simple, we need to reset the "atEnd" flag once seekBefore op is
> successful. You can undo my HalfStoreFileReader change and run the new added
> case to see NPE.
> ps: I also found some cases in TestHRegion missing the "@Test" annotation, so
> fix them in the same patch as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12779) SplitTransaction: Add metrics
[ https://issues.apache.org/jira/browse/HBASE-12779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-12779: -- Attachment: 12779-v2.txt Fixed metric names. > SplitTransaction: Add metrics > - > > Key: HBASE-12779 > URL: https://issues.apache.org/jira/browse/HBASE-12779 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl > Fix For: 2.0.0, 0.98.10, 1.0.1, 0.94.27 > > Attachments: 12779-v2.txt, 12779.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12782) ITBLL fails for me if generator does anything but 5M per maptask
stack created HBASE-12782: - Summary: ITBLL fails for me if generator does anything but 5M per maptask Key: HBASE-12782 URL: https://issues.apache.org/jira/browse/HBASE-12782 Project: HBase Issue Type: Bug Components: integration tests Affects Versions: 1.0.0 Reporter: stack Priority: Critical Anyone else seeing this? If I do an ITBLL with generator doing 5M rows per maptask, all is good -- verify passes. I've been running 5 servers and had one splot per server. So below works: HADOOP_CLASSPATH="/home/stack/conf_hbase:`/home/stack/hbase/bin/hbase classpath`" ./hadoop/bin/hadoop --config ~/conf_hadoop org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList --monkey serverKilling Generator 5 500 g1.tmp or if I double the map tasks, it works: HADOOP_CLASSPATH="/home/stack/conf_hbase:`/home/stack/hbase/bin/hbase classpath`" ./hadoop/bin/hadoop --config ~/conf_hadoop org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList --monkey serverKilling Generator 10 500 g2.tmp ...but if I change the 5M to 50M or 25M, Verify fails. Looking into it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12767) Fix a StoreFileScanner NPE in reverse scan flow
[
https://issues.apache.org/jira/browse/HBASE-12767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260876#comment-14260876
]
Hudson commented on HBASE-12767:
FAILURE: Integrated in HBase-0.98 #763 (See
[https://builds.apache.org/job/HBase-0.98/763/])
HBASE-12767 Fix a StoreFileScanner NPE in reverse scan flow (xieliang: rev
3197a631dbdf8cee7ba8fe1ea4864ac55fde5a43)
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
> Fix a StoreFileScanner NPE in reverse scan flow
> ---
>
> Key: HBASE-12767
> URL: https://issues.apache.org/jira/browse/HBASE-12767
> Project: HBase
> Issue Type: Bug
> Components: regionserver, Scanners
>Affects Versions: 1.0.0, 2.0.0, 0.98.9
>Reporter: Liang Xie
>Assignee: Liang Xie
> Fix For: 1.0.0, 2.0.0, 0.98.10, 1.1.0
>
> Attachments: HBASE-12767.txt
>
>
> we observed a NPE in our internal 0.94 branch prd env:
> ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekToPreviousRow(StoreFileScanner.java:409)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.backwardSeek(StoreFileScanner.java:467)
> at
> org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.seekScanners(ReversedStoreScanner.java:78)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:154)
> at
> org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.(ReversedStoreScanner.java:49)
> at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2216)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:3984)
> at
> org.apache.hadoop.hbase.regionserver.ReversedRegionScannerImpl.(ReversedRegionScannerImpl.java:43)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1851)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1837)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1812)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2752)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2715)
> After tracing, it turned out a boundary bug at HalfStoreFileReader. If we
> have a reverse scan request at "bottom" part, once the start row >= the last
> row at bottom part, the NPE will be encourtered, since we will invoke
> "next()", and "atEnd = true" will be called in HalfStoreFileReader, then when
> we want to do "seekBefore" or other sth in StoreFileScanner,
> StoreFileScanner.cur will call:
> {code}
> public Cell getKeyValue() {
> if (atEnd) return null;
> {code}
> The fix is simple, we need to reset the "atEnd" flag once seekBefore op is
> successful. You can undo my HalfStoreFileReader change and run the new added
> case to see NPE.
> ps: I also found some cases in TestHRegion missing the "@Test" annotation, so
> fix them in the same patch as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12776) SpliTransaction: Log number of files to be split
[ https://issues.apache.org/jira/browse/HBASE-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260875#comment-14260875 ] Hudson commented on HBASE-12776: FAILURE: Integrated in HBase-0.98 #763 (See [https://builds.apache.org/job/HBase-0.98/763/]) HBASE-12776 SpliTransaction: Log number of files to be split. (larsh: rev 3c5645f2b035af464ea0cf6181b241e1a002a501) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java > SpliTransaction: Log number of files to be split > > > Key: HBASE-12776 > URL: https://issues.apache.org/jira/browse/HBASE-12776 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 2.0.0, 0.98.10, 1.0.1, 1.1.0, 0.94.27 > > Attachments: 12765.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12776) SpliTransaction: Log number of files to be split
[ https://issues.apache.org/jira/browse/HBASE-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260873#comment-14260873 ] Hudson commented on HBASE-12776: SUCCESS: Integrated in HBase-1.0 #619 (See [https://builds.apache.org/job/HBase-1.0/619/]) HBASE-12776 SpliTransaction: Log number of files to be split. (larsh: rev 97c881a55579a9f439f2885ce5d51c4f0eb5a86d) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java > SpliTransaction: Log number of files to be split > > > Key: HBASE-12776 > URL: https://issues.apache.org/jira/browse/HBASE-12776 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 2.0.0, 0.98.10, 1.0.1, 1.1.0, 0.94.27 > > Attachments: 12765.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-5299) CatalogTracker.getMetaServerConnection() checks for root server connection and makes waitForMeta to go into infinite wait in region assignment flow.
[
https://issues.apache.org/jira/browse/HBASE-5299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-5299.
--
Resolution: Won't Fix
No CatalogTracker anymore in branch-1 and master so resolving as won't fix
> CatalogTracker.getMetaServerConnection() checks for root server connection
> and makes waitForMeta to go into infinite wait in region assignment flow.
>
>
> Key: HBASE-5299
> URL: https://issues.apache.org/jira/browse/HBASE-5299
> Project: HBase
> Issue Type: Bug
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
>
> RSA, RS B and RS C are 3 region servers.
> RS A -> META
> RS B -> ROOT
> RS C -> NON META and NON ROOT
> Kill RS B and wait for server shutdown handler to start.
> Start RS B again before assigning ROOT to RS C.
> Now the cluster will try to assign new regions to RS B.
> But as ROOT is not yet assigned the OpenRegionHandler.updateMeta will fail to
> update the regions just because ROOT is not online.
> {code}
> a87109263ed53e67158377a149c5a7be from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENING
> 2012-01-30 16:23:25,126 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:60020-0x1352e27539c0009 Attempting to transition node
> a87109263ed53e67158377a149c5a7be from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENING
> 2012-01-30 16:23:25,159 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:60020-0x1352e27539c0009 Successfully transitioned node
> a87109263ed53e67158377a149c5a7be from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENING
> 2012-01-30 16:23:35,385 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:60020-0x1352e27539c0009 Attempting to transition node
> a87109263ed53e67158377a149c5a7be from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENING
> 2012-01-30 16:23:35,449 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:60020-0x1352e27539c0009 Successfully transitioned node
> a87109263ed53e67158377a149c5a7be from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENING
> 2012-01-30 16:24:16,666 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:60020-0x1352e27539c0009 Attempting to transition node
> a87109263ed53e67158377a149c5a7be from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENING
> 2012-01-30 16:24:16,701 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:60020-0x1352e27539c0009 Successfully transitioned node
> a87109263ed53e67158377a149c5a7be from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENING
> 2012-01-30 16:24:20,788 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Interrupting
> thread Thread[PostOpenDeployTasks:a87109263ed53e67158377a149c5a7be,5,main]
> 2012-01-30 16:24:30,699 WARN
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Exception
> running postOpenDeployTasks; region=a87109263ed53e67158377a149c5a7be
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Interrupted
> at
> org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMetaServerConnectionDefault(CatalogTracker.java:439)
> at
> org.apache.hadoop.hbase.catalog.MetaEditor.updateRegionLocation(MetaEditor.java:142)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1382)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler$PostOpenDeployTasksThread.run(OpenRegionHandler.java:221)
> {code}
> So we need to wait for TM to assign the regions again.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12772) TestPerColumnFamilyFlush failing
[
https://issues.apache.org/jira/browse/HBASE-12772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260868#comment-14260868
]
stack commented on HBASE-12772:
---
Thanks [~dimaspivak] Was unable to repro locally. Looking at your log, its
same issue.. namespace table edit is oldest outstanding so when we hit the WAL
max count, that is the region that gets flushed, not the one the test is
expecting. Addendum coming.
> TestPerColumnFamilyFlush failing
>
>
> Key: HBASE-12772
> URL: https://issues.apache.org/jira/browse/HBASE-12772
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.0.0
>Reporter: stack
> Attachments: 0001-HBASE-12772-TestPerColumnFamilyFlush-failing.patch
>
>
> On internal rig see this failing in two places:
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling
> Failing for the past 1 build (Since Failed#653 )
> Took 9 sec.
> Error Message
> expected:<424> but was:<205744>
> Stacktrace
> java.lang.AssertionError: expected:<424> but was:<205744>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling(TestPerColumnFamilyFlush.java:483)
> {code}
> and
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay
> Failing for the past 1 build (Since Failed#653 )
> Took 25 ms.
> Error Message
> A mini-cluster is already running
> Stacktrace
> java.lang.IllegalStateException: A mini-cluster is already running
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:921)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:812)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:794)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:781)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplay(TestPerColumnFamilyFlush.java:337)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay(TestPerColumnFamilyFlush.java:418)
> {code}
> Opening issue to keep an eye on these tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12776) SpliTransaction: Log number of files to be split
[ https://issues.apache.org/jira/browse/HBASE-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260865#comment-14260865 ] Hudson commented on HBASE-12776: FAILURE: Integrated in HBase-1.1 #34 (See [https://builds.apache.org/job/HBase-1.1/34/]) HBASE-12776 SpliTransaction: Log number of files to be split. (larsh: rev 6a7c619e1e841dad5b570a269d05cabca88e6232) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java > SpliTransaction: Log number of files to be split > > > Key: HBASE-12776 > URL: https://issues.apache.org/jira/browse/HBASE-12776 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 2.0.0, 0.98.10, 1.0.1, 1.1.0, 0.94.27 > > Attachments: 12765.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12776) SpliTransaction: Log number of files to be split
[ https://issues.apache.org/jira/browse/HBASE-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260857#comment-14260857 ] Hudson commented on HBASE-12776: SUCCESS: Integrated in HBase-TRUNK #5975 (See [https://builds.apache.org/job/HBase-TRUNK/5975/]) HBASE-12776 SpliTransaction: Log number of files to be split. (larsh: rev 0513a21dc8b86f57b5a6c1b742904821632f77f7) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java > SpliTransaction: Log number of files to be split > > > Key: HBASE-12776 > URL: https://issues.apache.org/jira/browse/HBASE-12776 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 2.0.0, 0.98.10, 1.0.1, 1.1.0, 0.94.27 > > Attachments: 12765.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12768) Support enable cache_data_on_write in Shell while creating table
[ https://issues.apache.org/jira/browse/HBASE-12768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260856#comment-14260856 ] Hudson commented on HBASE-12768: SUCCESS: Integrated in HBase-TRUNK #5975 (See [https://builds.apache.org/job/HBase-TRUNK/5975/]) HBASE-12768 - Support enable cache_data_on_write in Shell while creating (ramkrishna: rev 0d094d877b5224d35c2721ae572eea05e0410a00) * hbase-shell/src/main/ruby/hbase/admin.rb > Support enable cache_data_on_write in Shell while creating table > > > Key: HBASE-12768 > URL: https://issues.apache.org/jira/browse/HBASE-12768 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 1.0.0, 2.0.0, 0.94.27 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 1.0.0, 2.0.0, 0.98.10 > > Attachments: HBASE-12768.patch > > > A simple approach to support cache_data_on_write while creating table in > shell. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12776) SpliTransaction: Log number of files to be split
[ https://issues.apache.org/jira/browse/HBASE-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260847#comment-14260847 ] Hudson commented on HBASE-12776: SUCCESS: Integrated in HBase-0.94-JDK7 #218 (See [https://builds.apache.org/job/HBase-0.94-JDK7/218/]) HBASE-12776 SpliTransaction: Log number of files to be split. (larsh: rev 403d4eb4d295d22611699fea8acd055ea9df8ed0) * src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java > SpliTransaction: Log number of files to be split > > > Key: HBASE-12776 > URL: https://issues.apache.org/jira/browse/HBASE-12776 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 2.0.0, 0.98.10, 1.0.1, 1.1.0, 0.94.27 > > Attachments: 12765.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12776) SpliTransaction: Log number of files to be split
[ https://issues.apache.org/jira/browse/HBASE-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260843#comment-14260843 ] Hudson commented on HBASE-12776: FAILURE: Integrated in HBase-0.94 #1450 (See [https://builds.apache.org/job/HBase-0.94/1450/]) HBASE-12776 SpliTransaction: Log number of files to be split. (larsh: rev 403d4eb4d295d22611699fea8acd055ea9df8ed0) * src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java > SpliTransaction: Log number of files to be split > > > Key: HBASE-12776 > URL: https://issues.apache.org/jira/browse/HBASE-12776 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 2.0.0, 0.98.10, 1.0.1, 1.1.0, 0.94.27 > > Attachments: 12765.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12776) SpliTransaction: Log number of files to be split
[ https://issues.apache.org/jira/browse/HBASE-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260842#comment-14260842 ] Hudson commented on HBASE-12776: SUCCESS: Integrated in HBase-0.94-security #563 (See [https://builds.apache.org/job/HBase-0.94-security/563/]) HBASE-12776 SpliTransaction: Log number of files to be split. (larsh: rev 403d4eb4d295d22611699fea8acd055ea9df8ed0) * src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java > SpliTransaction: Log number of files to be split > > > Key: HBASE-12776 > URL: https://issues.apache.org/jira/browse/HBASE-12776 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 2.0.0, 0.98.10, 1.0.1, 1.1.0, 0.94.27 > > Attachments: 12765.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12781) Listen port will bind always to the passed command line address
Pankaj Kumar created HBASE-12781:
Summary: Listen port will bind always to the passed command line
address
Key: HBASE-12781
URL: https://issues.apache.org/jira/browse/HBASE-12781
Project: HBase
Issue Type: Bug
Components: Thrift
Affects Versions: 0.98.3
Reporter: Pankaj Kumar
Assignee: Pankaj Kumar
In Thrift server, listen port will bind always to the address which is passed
through command line argument.
--
InetSocketAddress inetSocketAddress = bindToPort(cmd.getOptionValue("bind"),
listenPort);
-
private static InetSocketAddress bindToPort(String bindValue, int listenPort)
throws UnknownHostException {
try {
if (bindValue == null) {
return new InetSocketAddress(listenPort);
} else {
return new InetSocketAddress(InetAddress.getByName(bindValue),
listenPort);
}
} catch (UnknownHostException e) {
throw new RuntimeException("Could not bind to provided ip address", e);
}
}
In case when bind address is not passed through argument then it is binding
with any local address. It should read hbase.thrift.info.bindAddress value
from configuration first.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12712) skipLargeFiles in minor compact but not in major compact
[ https://issues.apache.org/jira/browse/HBASE-12712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260837#comment-14260837 ] Liu Junhong commented on HBASE-12712: - I need a major compaction triggered by compactionChecker even though i set hbase.hstore.compaction.max.size. In this case majorCompaction should be true when the function isMajorCompaction returns true. > skipLargeFiles in minor compact but not in major compact > > > Key: HBASE-12712 > URL: https://issues.apache.org/jira/browse/HBASE-12712 > Project: HBase > Issue Type: New Feature > Components: Compaction >Affects Versions: 0.98.6 >Reporter: Liu Junhong > Labels: newbie, patch > Fix For: 0.98.6 > > Attachments: compact.diff > > Original Estimate: 72h > Remaining Estimate: 72h > > Here is my case. After repeatedly minor compaction, the size of storefile is > very large. Compaction with large storefile will waste much bandwidth, so i > use the “hbase.hstore.compaction.max.size” to skip this case. But after use > this config, i find that major compaction will be skipped forever when i read > the source code and the deletes and muti-versions data my waste storage. So i > had to modify the code. > Now i'm try to submit my patch.But my patch is not perfect. I think there > should be an other config to determine if the large size storefile should > join major compaction in HColumnDescriptor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12767) Fix a StoreFileScanner NPE in reverse scan flow
[
https://issues.apache.org/jira/browse/HBASE-12767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260832#comment-14260832
]
Hudson commented on HBASE-12767:
FAILURE: Integrated in HBase-1.0 #618 (See
[https://builds.apache.org/job/HBase-1.0/618/])
HBASE-12767 Fix a StoreFileScanner NPE in reverse scan flow (xieliang: rev
92c71413564eecb05d8e8a93044e769d46a17a02)
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
> Fix a StoreFileScanner NPE in reverse scan flow
> ---
>
> Key: HBASE-12767
> URL: https://issues.apache.org/jira/browse/HBASE-12767
> Project: HBase
> Issue Type: Bug
> Components: regionserver, Scanners
>Affects Versions: 1.0.0, 2.0.0, 0.98.9
>Reporter: Liang Xie
>Assignee: Liang Xie
> Fix For: 1.0.0, 2.0.0, 0.98.10, 1.1.0
>
> Attachments: HBASE-12767.txt
>
>
> we observed a NPE in our internal 0.94 branch prd env:
> ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekToPreviousRow(StoreFileScanner.java:409)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.backwardSeek(StoreFileScanner.java:467)
> at
> org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.seekScanners(ReversedStoreScanner.java:78)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:154)
> at
> org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.(ReversedStoreScanner.java:49)
> at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2216)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:3984)
> at
> org.apache.hadoop.hbase.regionserver.ReversedRegionScannerImpl.(ReversedRegionScannerImpl.java:43)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1851)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1837)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1812)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2752)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2715)
> After tracing, it turned out a boundary bug at HalfStoreFileReader. If we
> have a reverse scan request at "bottom" part, once the start row >= the last
> row at bottom part, the NPE will be encourtered, since we will invoke
> "next()", and "atEnd = true" will be called in HalfStoreFileReader, then when
> we want to do "seekBefore" or other sth in StoreFileScanner,
> StoreFileScanner.cur will call:
> {code}
> public Cell getKeyValue() {
> if (atEnd) return null;
> {code}
> The fix is simple, we need to reset the "atEnd" flag once seekBefore op is
> successful. You can undo my HalfStoreFileReader change and run the new added
> case to see NPE.
> ps: I also found some cases in TestHRegion missing the "@Test" annotation, so
> fix them in the same patch as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12767) Fix a StoreFileScanner NPE in reverse scan flow
[
https://issues.apache.org/jira/browse/HBASE-12767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260830#comment-14260830
]
Hudson commented on HBASE-12767:
FAILURE: Integrated in HBase-1.1 #33 (See
[https://builds.apache.org/job/HBase-1.1/33/])
HBASE-12767 Fix a StoreFileScanner NPE in reverse scan flow (xieliang: rev
619f94f113f9952995c61c851b379aec72bf52bb)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java
> Fix a StoreFileScanner NPE in reverse scan flow
> ---
>
> Key: HBASE-12767
> URL: https://issues.apache.org/jira/browse/HBASE-12767
> Project: HBase
> Issue Type: Bug
> Components: regionserver, Scanners
>Affects Versions: 1.0.0, 2.0.0, 0.98.9
>Reporter: Liang Xie
>Assignee: Liang Xie
> Fix For: 1.0.0, 2.0.0, 0.98.10, 1.1.0
>
> Attachments: HBASE-12767.txt
>
>
> we observed a NPE in our internal 0.94 branch prd env:
> ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekToPreviousRow(StoreFileScanner.java:409)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.backwardSeek(StoreFileScanner.java:467)
> at
> org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.seekScanners(ReversedStoreScanner.java:78)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:154)
> at
> org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.(ReversedStoreScanner.java:49)
> at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2216)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:3984)
> at
> org.apache.hadoop.hbase.regionserver.ReversedRegionScannerImpl.(ReversedRegionScannerImpl.java:43)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1851)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1837)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1812)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2752)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2715)
> After tracing, it turned out a boundary bug at HalfStoreFileReader. If we
> have a reverse scan request at "bottom" part, once the start row >= the last
> row at bottom part, the NPE will be encourtered, since we will invoke
> "next()", and "atEnd = true" will be called in HalfStoreFileReader, then when
> we want to do "seekBefore" or other sth in StoreFileScanner,
> StoreFileScanner.cur will call:
> {code}
> public Cell getKeyValue() {
> if (atEnd) return null;
> {code}
> The fix is simple, we need to reset the "atEnd" flag once seekBefore op is
> successful. You can undo my HalfStoreFileReader change and run the new added
> case to see NPE.
> ps: I also found some cases in TestHRegion missing the "@Test" annotation, so
> fix them in the same patch as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12767) Fix a StoreFileScanner NPE in reverse scan flow
[
https://issues.apache.org/jira/browse/HBASE-12767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260817#comment-14260817
]
Hudson commented on HBASE-12767:
SUCCESS: Integrated in HBase-TRUNK #5974 (See
[https://builds.apache.org/job/HBase-TRUNK/5974/])
HBASE-12767 Fix a StoreFileScanner NPE in reverse scan flow (xieliang: rev
c24810eebe1f52bbe42bd0f4e0bf512425295aad)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java
> Fix a StoreFileScanner NPE in reverse scan flow
> ---
>
> Key: HBASE-12767
> URL: https://issues.apache.org/jira/browse/HBASE-12767
> Project: HBase
> Issue Type: Bug
> Components: regionserver, Scanners
>Affects Versions: 1.0.0, 2.0.0, 0.98.9
>Reporter: Liang Xie
>Assignee: Liang Xie
> Fix For: 1.0.0, 2.0.0, 0.98.10, 1.1.0
>
> Attachments: HBASE-12767.txt
>
>
> we observed a NPE in our internal 0.94 branch prd env:
> ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekToPreviousRow(StoreFileScanner.java:409)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.backwardSeek(StoreFileScanner.java:467)
> at
> org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.seekScanners(ReversedStoreScanner.java:78)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:154)
> at
> org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.(ReversedStoreScanner.java:49)
> at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2216)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:3984)
> at
> org.apache.hadoop.hbase.regionserver.ReversedRegionScannerImpl.(ReversedRegionScannerImpl.java:43)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1851)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1837)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1812)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2752)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2715)
> After tracing, it turned out a boundary bug at HalfStoreFileReader. If we
> have a reverse scan request at "bottom" part, once the start row >= the last
> row at bottom part, the NPE will be encourtered, since we will invoke
> "next()", and "atEnd = true" will be called in HalfStoreFileReader, then when
> we want to do "seekBefore" or other sth in StoreFileScanner,
> StoreFileScanner.cur will call:
> {code}
> public Cell getKeyValue() {
> if (atEnd) return null;
> {code}
> The fix is simple, we need to reset the "atEnd" flag once seekBefore op is
> successful. You can undo my HalfStoreFileReader change and run the new added
> case to see NPE.
> ps: I also found some cases in TestHRegion missing the "@Test" annotation, so
> fix them in the same patch as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12779) SplitTransaction: Add metrics
[ https://issues.apache.org/jira/browse/HBASE-12779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260797#comment-14260797 ] Lars Hofhansl commented on HBASE-12779: --- (after I test this, of course) > SplitTransaction: Add metrics > - > > Key: HBASE-12779 > URL: https://issues.apache.org/jira/browse/HBASE-12779 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl > Fix For: 2.0.0, 0.98.10, 1.0.1, 0.94.27 > > Attachments: 12779.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12779) SplitTransaction: Add metrics
[ https://issues.apache.org/jira/browse/HBASE-12779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260793#comment-14260793 ] Lars Hofhansl commented on HBASE-12779: --- Cool... I'll add the file count metric and then create a 0.98 patch. Actually for the file count metric it's a bit less obvious. # should it report the requested hfiles to split? # successfully split hfiles? # successful files separated by what goes to daughter A vs B? # all of the above? I guess #4 makes the most sense. Maybe commit this one first as is, then discuss what to do with split file counts in another jira. This would allow tracking splits as they happen per region server and to monitor the split queue. Would also be nice to track the provenance of the regions. That was suggested before, but it was never clear where to store this information... Another issue... > SplitTransaction: Add metrics > - > > Key: HBASE-12779 > URL: https://issues.apache.org/jira/browse/HBASE-12779 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl > Fix For: 2.0.0, 0.98.10, 1.0.1, 0.94.27 > > Attachments: 12779.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12776) SpliTransaction: Log number of files to be split
[ https://issues.apache.org/jira/browse/HBASE-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-12776: -- Fix Version/s: 1.0.1 > SpliTransaction: Log number of files to be split > > > Key: HBASE-12776 > URL: https://issues.apache.org/jira/browse/HBASE-12776 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 2.0.0, 0.98.10, 1.0.1, 1.1.0, 0.94.27 > > Attachments: 12765.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12776) SpliTransaction: Log number of files to be split
[ https://issues.apache.org/jira/browse/HBASE-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-12776: -- Fix Version/s: (was: 1.0.1) 1.1.0 > SpliTransaction: Log number of files to be split > > > Key: HBASE-12776 > URL: https://issues.apache.org/jira/browse/HBASE-12776 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 2.0.0, 0.98.10, 1.1.0, 0.94.27 > > Attachments: 12765.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-12776) SpliTransaction: Log number of files to be split
[ https://issues.apache.org/jira/browse/HBASE-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-12776. --- Resolution: Fixed Assignee: Lars Hofhansl Hadoop Flags: Reviewed Committed to all branches. (the 0.94 version only has the first part as it does not keep track of the daughter a/b counts, and it did not make sense to add that part) > SpliTransaction: Log number of files to be split > > > Key: HBASE-12776 > URL: https://issues.apache.org/jira/browse/HBASE-12776 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 2.0.0, 0.98.10, 1.0.1, 0.94.27 > > Attachments: 12765.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-6048) Table Scan is failing if offheap cache enabled
[
https://issues.apache.org/jira/browse/HBASE-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan resolved HBASE-6048.
---
Resolution: Duplicate
Duplicate. Chunhui's fix has solved this already.
> Table Scan is failing if offheap cache enabled
> --
>
> Key: HBASE-6048
> URL: https://issues.apache.org/jira/browse/HBASE-6048
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
> Attachments: hbase-6048-94.patch
>
>
> Table Scan is failing if offheap cache enabled.
> {noformat}
> 2012-05-18 20:03:38,446 DEBUG org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
> Initialized with CacheConfig:enabled [cacheDataOnRead=true]
> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> [cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false]
> 2012-05-18 20:03:38,446 INFO org.apache.hadoop.hbase.regionserver.StoreFile:
> Delete Family Bloom filter type for
> hdfs://10.18.40.217:9000/hbase/ufdr/1d4656fd417a07c9171a38b8f4d08510/.tmp/03742024b28f443bb63cfc338d4ca422:
> CompoundBloomFilterWriter
> 2012-05-18 20:04:25,576 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache:
> Block cache LRU eviction started; Attempting to free 120.57 MB of
> total=1020.57 MB
> 2012-05-18 20:04:25,655 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache:
> Block cache LRU eviction completed; freed=120.82 MB, total=907.89 MB,
> single=1012.11 MB, multi=6.12 MB, memory=0 KB
> 2012-05-18 20:04:25,733 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner
> java.lang.IllegalStateException: Schema metrics requested before table/CF
> name initialization: {"tableName":"null","cfName":"null"}
> at
> org.apache.hadoop.hbase.regionserver.metrics.SchemaConfigured.getSchemaMetrics(SchemaConfigured.java:182)
> at
> org.apache.hadoop.hbase.io.hfile.LruBlockCache.updateSizeMetrics(LruBlockCache.java:310)
> at
> org.apache.hadoop.hbase.io.hfile.LruBlockCache.cacheBlock(LruBlockCache.java:274)
> at
> org.apache.hadoop.hbase.io.hfile.LruBlockCache.cacheBlock(LruBlockCache.java:293)
> at
> org.apache.hadoop.hbase.io.hfile.DoubleBlockCache.getBlock(DoubleBlockCache.java:102)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:296)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:213)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:455)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:475)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:226)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:145)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:130)
> at
> org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2001)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:3274)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1604)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1596)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1572)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2310)
> at sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1376)
> 2012-05-18 20:04:25,828 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-8132) Master rebuild() should check for regioninfo count before ENABLING/DISABLING partial tables
[ https://issues.apache.org/jira/browse/HBASE-8132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan resolved HBASE-8132. --- Resolution: Later > Master rebuild() should check for regioninfo count before ENABLING/DISABLING > partial tables > --- > > Key: HBASE-8132 > URL: https://issues.apache.org/jira/browse/HBASE-8132 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > > After snapshots addition the adding of regioninfo into the META is more of > atomic nature. This solves the problem that was raised in HBASE-5583. > But when we do a restart of master or failover of master we do not validate > the regioninfo that was added to META if it was same as that was mentioned > while creation of table. > This validation may be needed so that a master restart will enable to create > the exact number of regions as given during the create table splitkeys. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-8578) Improve flushCache to distinguish flush and compaction needed scenarios
[ https://issues.apache.org/jira/browse/HBASE-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan resolved HBASE-8578. --- Resolution: Later NEed to see the latest code if still exists. Resolving as 'Later' > Improve flushCache to distinguish flush and compaction needed scenarios > --- > > Key: HBASE-8578 > URL: https://issues.apache.org/jira/browse/HBASE-8578 > Project: HBase > Issue Type: Improvement >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > > Moving out an ill-specified issue that does not look like it is being worked > on. Move back if you think otherwise Ram. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-4951) master process can not be stopped when it is initializing
[ https://issues.apache.org/jira/browse/HBASE-4951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan resolved HBASE-4951. --- Resolution: Won't Fix 0.90.x is not active. > master process can not be stopped when it is initializing > - > > Key: HBASE-4951 > URL: https://issues.apache.org/jira/browse/HBASE-4951 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.3 >Reporter: xufeng >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Attachments: HBASE-4951.patch, HBASE-4951_branch.patch > > > It is easy to reproduce by following step: > step1:start master process.(do not start regionserver process in the cluster). > the master will wait the regionserver to check in: > org.apache.hadoop.hbase.master.ServerManager: Waiting on regionserver(s) to > checkin > step2:stop the master by sh command bin/hbase master stop > result:the master process will never die because catalogTracker.waitForRoot() > method will block unitl the root region assigned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-5816) Balancer and ServerShutdownHandler concurrently reassign the same region
[
https://issues.apache.org/jira/browse/HBASE-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan resolved HBASE-5816.
---
Resolution: Duplicate
AM rewrite has done a lot of changes to handle these cases.
> Balancer and ServerShutdownHandler concurrently reassign the same region
>
>
> Key: HBASE-5816
> URL: https://issues.apache.org/jira/browse/HBASE-5816
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.90.6
>Reporter: Maryann Xue
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Attachments: HBASE-5816.patch
>
>
> The first assign thread exits with success after updating the RegionState to
> PENDING_OPEN, while the second assign follows immediately into "assign" and
> fails the RegionState check in setOfflineInZooKeeper(). This causes the
> master to abort.
> In the below case, the two concurrent assigns occurred when AM tried to
> assign a region to a dying/dead RS, and meanwhile the ShutdownServerHandler
> tried to assign this region (from the region plan) spontaneously.
> {code}
> 2012-04-17 05:44:57,648 INFO org.apache.hadoop.hbase.master.HMaster: balance
> hri=TABLE_ORDER_CUSTOMER,,1334017820846.fe38fe31caf40b6e607a3e6bbed6404b.,
> src=hadoop05.sh.intel.com,60020,1334544902186,
> dest=xmlqa-clv16.sh.intel.com,60020,1334612497253
> 2012-04-17 05:44:57,648 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Starting unassignment of
> region TABLE_ORDER_CUSTOMER,,1334017820846.fe38fe31caf40b6e607a3e6bbed6404b.
> (offlining)
> 2012-04-17 05:44:57,648 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Sent CLOSE to
> serverName=hadoop05.sh.intel.com,60020,1334544902186, load=(requests=0,
> regions=0, usedHeap=0, maxHeap=0) for region
> TABLE_ORDER_CUSTOMER,,1334017820846.fe38fe31caf40b6e607a3e6bbed6404b.
> 2012-04-17 05:44:57,666 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling new unassigned
> node: /hbase/unassigned/fe38fe31caf40b6e607a3e6bbed6404b
> (region=TABLE_ORDER_CUSTOMER,,1334017820846.fe38fe31caf40b6e607a3e6bbed6404b.,
> server=hadoop05.sh.intel.com,60020,1334544902186, state=RS_ZK_REGION_CLOSING)
> 2012-04-17 05:52:58,984 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE;
> was=TABLE_ORDER_CUSTOMER,,1334017820846.fe38fe31caf40b6e607a3e6bbed6404b.
> state=CLOSED, ts=1334612697672,
> server=hadoop05.sh.intel.com,60020,1334544902186
> 2012-04-17 05:52:58,984 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> master:6-0x236b912e9b3000e Creating (or updating) unassigned node for
> fe38fe31caf40b6e607a3e6bbed6404b with OFFLINE state
> 2012-04-17 05:52:59,096 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Using pre-existing plan for
> region TABLE_ORDER_CUSTOMER,,1334017820846.fe38fe31caf40b6e607a3e6bbed6404b.;
> plan=hri=TABLE_ORDER_CUSTOMER,,1334017820846.fe38fe31caf40b6e607a3e6bbed6404b.,
> src=hadoop05.sh.intel.com,60020,1334544902186,
> dest=xmlqa-clv16.sh.intel.com,60020,1334612497253
> 2012-04-17 05:52:59,096 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Assigning region
> TABLE_ORDER_CUSTOMER,,1334017820846.fe38fe31caf40b6e607a3e6bbed6404b. to
> xmlqa-clv16.sh.intel.com,60020,1334612497253
> 2012-04-17 05:54:19,159 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE;
> was=TABLE_ORDER_CUSTOMER,,1334017820846.fe38fe31caf40b6e607a3e6bbed6404b.
> state=PENDING_OPEN, ts=1334613179096,
> server=xmlqa-clv16.sh.intel.com,60020,1334612497253
> 2012-04-17 05:54:59,033 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Failed assignment of
> TABLE_ORDER_CUSTOMER,,1334017820846.fe38fe31caf40b6e607a3e6bbed6404b. to
> serverName=xmlqa-clv16.sh.intel.com,60020,1334612497253, load=(requests=0,
> regions=0, usedHeap=0, maxHeap=0), trying to assign elsewhere instead; retry=0
> java.net.SocketTimeoutException: Call to /10.239.47.87:60020 failed on socket
> timeout exception: java.net.SocketTimeoutException: 12 millis timeout
> while waiting for channel to be ready for read. ch :
> java.nio.channels.SocketChannel[connected local=/10.239.47.89:41302
> remote=/10.239.47.87:60020]
> at
> org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:805)
> at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:778)
> at
> org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:283)
> at $Proxy7.openRegion(Unknown Source)
> at
> org.apache.hadoop.hbase.master.ServerManager.sendRegionOpen(ServerManager.java:573)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1127)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assi
[jira] [Resolved] (HBASE-6273) HMasterInterface.isMasterRunning() requires clean up
[ https://issues.apache.org/jira/browse/HBASE-6273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan resolved HBASE-6273. --- Resolution: Won't Fix > HMasterInterface.isMasterRunning() requires clean up > > > Key: HBASE-6273 > URL: https://issues.apache.org/jira/browse/HBASE-6273 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 0.94.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > > This JIRA is in reference to JD's comments regarding the clean up needed in > isMasterRunning(). Refer to > https://issues.apache.org/jira/browse/HBASE-6240?focusedCommentId=13400772&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13400772 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12767) Fix a StoreFileScanner NPE in reverse scan flow
[
https://issues.apache.org/jira/browse/HBASE-12767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liang Xie updated HBASE-12767:
--
Hadoop Flags: Reviewed
> Fix a StoreFileScanner NPE in reverse scan flow
> ---
>
> Key: HBASE-12767
> URL: https://issues.apache.org/jira/browse/HBASE-12767
> Project: HBase
> Issue Type: Bug
> Components: regionserver, Scanners
>Affects Versions: 1.0.0, 2.0.0, 0.98.9
>Reporter: Liang Xie
>Assignee: Liang Xie
> Fix For: 1.0.0, 2.0.0, 0.98.10, 1.1.0
>
> Attachments: HBASE-12767.txt
>
>
> we observed a NPE in our internal 0.94 branch prd env:
> ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekToPreviousRow(StoreFileScanner.java:409)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.backwardSeek(StoreFileScanner.java:467)
> at
> org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.seekScanners(ReversedStoreScanner.java:78)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:154)
> at
> org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.(ReversedStoreScanner.java:49)
> at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2216)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:3984)
> at
> org.apache.hadoop.hbase.regionserver.ReversedRegionScannerImpl.(ReversedRegionScannerImpl.java:43)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1851)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1837)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1812)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2752)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2715)
> After tracing, it turned out a boundary bug at HalfStoreFileReader. If we
> have a reverse scan request at "bottom" part, once the start row >= the last
> row at bottom part, the NPE will be encourtered, since we will invoke
> "next()", and "atEnd = true" will be called in HalfStoreFileReader, then when
> we want to do "seekBefore" or other sth in StoreFileScanner,
> StoreFileScanner.cur will call:
> {code}
> public Cell getKeyValue() {
> if (atEnd) return null;
> {code}
> The fix is simple, we need to reset the "atEnd" flag once seekBefore op is
> successful. You can undo my HalfStoreFileReader change and run the new added
> case to see NPE.
> ps: I also found some cases in TestHRegion missing the "@Test" annotation, so
> fix them in the same patch as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12767) Fix a StoreFileScanner NPE in reverse scan flow
[
https://issues.apache.org/jira/browse/HBASE-12767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liang Xie updated HBASE-12767:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Fix a StoreFileScanner NPE in reverse scan flow
> ---
>
> Key: HBASE-12767
> URL: https://issues.apache.org/jira/browse/HBASE-12767
> Project: HBase
> Issue Type: Bug
> Components: regionserver, Scanners
>Affects Versions: 1.0.0, 2.0.0, 0.98.9
>Reporter: Liang Xie
>Assignee: Liang Xie
> Fix For: 1.0.0, 2.0.0, 0.98.10, 1.1.0
>
> Attachments: HBASE-12767.txt
>
>
> we observed a NPE in our internal 0.94 branch prd env:
> ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekToPreviousRow(StoreFileScanner.java:409)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.backwardSeek(StoreFileScanner.java:467)
> at
> org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.seekScanners(ReversedStoreScanner.java:78)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:154)
> at
> org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.(ReversedStoreScanner.java:49)
> at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2216)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:3984)
> at
> org.apache.hadoop.hbase.regionserver.ReversedRegionScannerImpl.(ReversedRegionScannerImpl.java:43)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1851)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1837)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1812)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2752)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2715)
> After tracing, it turned out a boundary bug at HalfStoreFileReader. If we
> have a reverse scan request at "bottom" part, once the start row >= the last
> row at bottom part, the NPE will be encourtered, since we will invoke
> "next()", and "atEnd = true" will be called in HalfStoreFileReader, then when
> we want to do "seekBefore" or other sth in StoreFileScanner,
> StoreFileScanner.cur will call:
> {code}
> public Cell getKeyValue() {
> if (atEnd) return null;
> {code}
> The fix is simple, we need to reset the "atEnd" flag once seekBefore op is
> successful. You can undo my HalfStoreFileReader change and run the new added
> case to see NPE.
> ps: I also found some cases in TestHRegion missing the "@Test" annotation, so
> fix them in the same patch as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-8193) HBaseAdmin.isTableAvailable should also check if the table is Enabled after successful creation
[ https://issues.apache.org/jira/browse/HBASE-8193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan resolved HBASE-8193. --- Resolution: Won't Fix > HBaseAdmin.isTableAvailable should also check if the table is Enabled after > successful creation > --- > > Key: HBASE-8193 > URL: https://issues.apache.org/jira/browse/HBASE-8193 > Project: HBase > Issue Type: Improvement >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > > Currently HBaseAdmin.isTableAvailable() only checks in META to say if a > table is available or not. It should also check with the zkTable state if it > is ENABLED before returning true. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-7962) Native folder not available under lib folder in the HBase trunk deployment
[ https://issues.apache.org/jira/browse/HBASE-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan resolved HBASE-7962. --- Resolution: Not a Problem > Native folder not available under lib folder in the HBase trunk deployment > -- > > Key: HBASE-7962 > URL: https://issues.apache.org/jira/browse/HBASE-7962 > Project: HBase > Issue Type: Bug >Affects Versions: 0.95.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > > I was trying to install Snappy. Found this link. > http://www.spaggiari.org/index.php/hbase/how-to-install-snappy-with#.US7JNzDX_D4. > As per this link there is a native folder under lib directory in the HBase > installation path and it is available in 0.94. > But when i install HBase trunk i could not see one > If am not finding this in the correct place, pls invalidate the issue. Will > try figuring out what could be the problem -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-5323) Need to handle assertion error while splitting log through ServerShutDownHandler by shutting down the master
[
https://issues.apache.org/jira/browse/HBASE-5323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan resolved HBASE-5323.
---
Resolution: Won't Fix
0.90.x version is no more active. Resolving as 'Won't fix'.
> Need to handle assertion error while splitting log through
> ServerShutDownHandler by shutting down the master
>
>
> Key: HBASE-5323
> URL: https://issues.apache.org/jira/browse/HBASE-5323
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.90.5
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-5323.patch, HBASE-5323.patch
>
>
> We know that while parsing the HLog we expect the proper length from HDFS.
> In WALReaderFSDataInputStream
> {code}
> assert(realLength >= this.length);
> {code}
> We are trying to come out if the above condition is not satisfied. But if
> SSH.splitLog() gets this problem then it lands in the run method of
> EventHandler. This kills the SSH thread and so further assignment does not
> happen. If ROOT and META are to be assigned they cannot be.
> I think in this condition we abort the master by catching such exceptions.
> Please do suggest.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12767) Fix a StoreFileScanner NPE in reverse scan flow
[
https://issues.apache.org/jira/browse/HBASE-12767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liang Xie updated HBASE-12767:
--
Fix Version/s: 1.1.0
0.98.10
2.0.0
1.0.0
> Fix a StoreFileScanner NPE in reverse scan flow
> ---
>
> Key: HBASE-12767
> URL: https://issues.apache.org/jira/browse/HBASE-12767
> Project: HBase
> Issue Type: Bug
> Components: regionserver, Scanners
>Affects Versions: 1.0.0, 2.0.0, 0.98.9
>Reporter: Liang Xie
>Assignee: Liang Xie
> Fix For: 1.0.0, 2.0.0, 0.98.10, 1.1.0
>
> Attachments: HBASE-12767.txt
>
>
> we observed a NPE in our internal 0.94 branch prd env:
> ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekToPreviousRow(StoreFileScanner.java:409)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.backwardSeek(StoreFileScanner.java:467)
> at
> org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.seekScanners(ReversedStoreScanner.java:78)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:154)
> at
> org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.(ReversedStoreScanner.java:49)
> at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2216)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:3984)
> at
> org.apache.hadoop.hbase.regionserver.ReversedRegionScannerImpl.(ReversedRegionScannerImpl.java:43)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1851)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1837)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1812)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2752)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2715)
> After tracing, it turned out a boundary bug at HalfStoreFileReader. If we
> have a reverse scan request at "bottom" part, once the start row >= the last
> row at bottom part, the NPE will be encourtered, since we will invoke
> "next()", and "atEnd = true" will be called in HalfStoreFileReader, then when
> we want to do "seekBefore" or other sth in StoreFileScanner,
> StoreFileScanner.cur will call:
> {code}
> public Cell getKeyValue() {
> if (atEnd) return null;
> {code}
> The fix is simple, we need to reset the "atEnd" flag once seekBefore op is
> successful. You can undo my HalfStoreFileReader change and run the new added
> case to see NPE.
> ps: I also found some cases in TestHRegion missing the "@Test" annotation, so
> fix them in the same patch as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-4060) Making region assignment more robust
[ https://issues.apache.org/jira/browse/HBASE-4060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan resolved HBASE-4060. --- Resolution: Duplicate AM rewrite as already done this robustness. Resolving as 'duplicate'. > Making region assignment more robust > > > Key: HBASE-4060 > URL: https://issues.apache.org/jira/browse/HBASE-4060 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: ramkrishna.s.vasudevan > > From Eran Kutner: > My concern is that the region allocation process seems to rely too much on > timing considerations and doesn't seem to take enough measures to guarantee > conflicts do not occur. I understand that in a distributed environment, when > you don't get a timely response from a remote machine you can't know for > sure if it did or did not receive the request, however there are things that > can be done to mitigate this and reduce the conflict time significantly. For > example, when I run dbck it knows that some regions are multiply assigned, > the master could do the same and try to resolve the conflict. Another > approach would be to handle late responses, even if the response from the > remote machine arrives after it was assumed to be dead the master should > have enough information to know it had created a conflict by assigning the > region to another server. An even better solution, I think, is for the RS to > periodically test that it is indeed the rightful owner of every region it > holds and relinquish control over the region if it's not. > Obviously a state where two RSs hold the same region is pathological and can > lead to data loss, as demonstrated in my case. The system should be able to > actively protect itself against such a scenario. It probably doesn't need > saying but there is really nothing worse for a data storage system than data > loss. > In my case the problem didn't happen in the initial phase but after > disabling and enabling a table with about 12K regions. > For more background information, see 'Errors after major compaction' > discussion on u...@hbase.apache.org -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-5516) GZip leading to memory leak in 0.90. Fix similar to HBASE-5387 needed for 0.90.
[ https://issues.apache.org/jira/browse/HBASE-5516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan resolved HBASE-5516. --- Resolution: Won't Fix 0.90.x versions are obsolete. Resolving as 'won't fix'. > GZip leading to memory leak in 0.90. Fix similar to HBASE-5387 needed for > 0.90. > > > Key: HBASE-5516 > URL: https://issues.apache.org/jira/browse/HBASE-5516 > Project: HBase > Issue Type: Bug >Affects Versions: 0.90.5 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Attachments: HBASE-5516_2_0.90.patch, HBASE-5516_3_0.90.patch > > > Usage of GZip is leading to resident memory leak in 0.90. > We need to have something similar to HBASE-5387 in 0.90. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-5651) [findbugs] Address wait/notify synchronization/inconsistency in sync
[ https://issues.apache.org/jira/browse/HBASE-5651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan resolved HBASE-5651. --- Resolution: Later Resolving this as 'Later'. > [findbugs] Address wait/notify synchronization/inconsistency in sync > > > Key: HBASE-5651 > URL: https://issues.apache.org/jira/browse/HBASE-5651 > Project: HBase > Issue Type: Sub-task > Components: scripts >Reporter: Jonathan Hsieh >Assignee: ramkrishna.s.vasudevan > > See > https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_MT_CORRECTNESS > fix classes IS,LI,MWM, NN, SWL, UG, UW -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12779) SplitTransaction: Add metrics
[ https://issues.apache.org/jira/browse/HBASE-12779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260772#comment-14260772 ] Andrew Purtell commented on HBASE-12779: Easy. About the same as in hadoop2-compat with minor differences. > SplitTransaction: Add metrics > - > > Key: HBASE-12779 > URL: https://issues.apache.org/jira/browse/HBASE-12779 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl > Fix For: 2.0.0, 0.98.10, 1.0.1, 0.94.27 > > Attachments: 12779.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-5116) Remove double assignment of regions
[ https://issues.apache.org/jira/browse/HBASE-5116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan resolved HBASE-5116. --- Resolution: Won't Fix AM rewrite should have taken care of these issues. Resolving as 'Won't fix' > Remove double assignment of regions > --- > > Key: HBASE-5116 > URL: https://issues.apache.org/jira/browse/HBASE-5116 > Project: HBase > Issue Type: Bug >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > > As part of HBASE-5094 we found the possibility of doubly assignments of > regions. This JIRA is to avoid such double assignments. > The idea is to get the regions corresponding to an RS in the the META and > compare the regions online in the RS. Remove those regions from the online > list if they dont match. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-5299) CatalogTracker.getMetaServerConnection() checks for root server connection and makes waitForMeta to go into infinite wait in region assignment flow.
[
https://issues.apache.org/jira/browse/HBASE-5299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260771#comment-14260771
]
ramkrishna.s.vasudevan commented on HBASE-5299:
---
[~saint@gmail.com]
You still want to check this. Not sure if this problem is still there.
> CatalogTracker.getMetaServerConnection() checks for root server connection
> and makes waitForMeta to go into infinite wait in region assignment flow.
>
>
> Key: HBASE-5299
> URL: https://issues.apache.org/jira/browse/HBASE-5299
> Project: HBase
> Issue Type: Bug
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
>
> RSA, RS B and RS C are 3 region servers.
> RS A -> META
> RS B -> ROOT
> RS C -> NON META and NON ROOT
> Kill RS B and wait for server shutdown handler to start.
> Start RS B again before assigning ROOT to RS C.
> Now the cluster will try to assign new regions to RS B.
> But as ROOT is not yet assigned the OpenRegionHandler.updateMeta will fail to
> update the regions just because ROOT is not online.
> {code}
> a87109263ed53e67158377a149c5a7be from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENING
> 2012-01-30 16:23:25,126 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:60020-0x1352e27539c0009 Attempting to transition node
> a87109263ed53e67158377a149c5a7be from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENING
> 2012-01-30 16:23:25,159 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:60020-0x1352e27539c0009 Successfully transitioned node
> a87109263ed53e67158377a149c5a7be from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENING
> 2012-01-30 16:23:35,385 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:60020-0x1352e27539c0009 Attempting to transition node
> a87109263ed53e67158377a149c5a7be from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENING
> 2012-01-30 16:23:35,449 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:60020-0x1352e27539c0009 Successfully transitioned node
> a87109263ed53e67158377a149c5a7be from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENING
> 2012-01-30 16:24:16,666 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:60020-0x1352e27539c0009 Attempting to transition node
> a87109263ed53e67158377a149c5a7be from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENING
> 2012-01-30 16:24:16,701 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:60020-0x1352e27539c0009 Successfully transitioned node
> a87109263ed53e67158377a149c5a7be from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENING
> 2012-01-30 16:24:20,788 DEBUG
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Interrupting
> thread Thread[PostOpenDeployTasks:a87109263ed53e67158377a149c5a7be,5,main]
> 2012-01-30 16:24:30,699 WARN
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Exception
> running postOpenDeployTasks; region=a87109263ed53e67158377a149c5a7be
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Interrupted
> at
> org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMetaServerConnectionDefault(CatalogTracker.java:439)
> at
> org.apache.hadoop.hbase.catalog.MetaEditor.updateRegionLocation(MetaEditor.java:142)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1382)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler$PostOpenDeployTasksThread.run(OpenRegionHandler.java:221)
> {code}
> So we need to wait for TM to assign the regions again.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-4841) If I call split fast enough, while inserting, rows disappear.
[ https://issues.apache.org/jira/browse/HBASE-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan resolved HBASE-4841. --- Resolution: Later There have been lot of changes in the split related and AM code. So will resolve this as 'Later' > If I call split fast enough, while inserting, rows disappear. > -- > > Key: HBASE-4841 > URL: https://issues.apache.org/jira/browse/HBASE-4841 > Project: HBase > Issue Type: Bug >Reporter: Alex Newman >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Attachments: 1, log, log2 > > > I'll attach a unit test for this. Basically if you call split, while > inserting data you can get to the point to where the cluster becomes > unstable, or rows will disappear. The unit test gives you some flexibility > of: > - How many rows > - How wide the rows are > - The frequency of the split. > The default settings crash unit tests or cause the unit tests to fail on my > laptop. On my macbook air, i could actually turn down the number of total > rows, and the frequency of the splits which is surprising. I think this is > because the macbook air has much better IO than my backup acer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12779) SplitTransaction: Add metrics
[ https://issues.apache.org/jira/browse/HBASE-12779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260764#comment-14260764 ] Lars Hofhansl commented on HBASE-12779: --- I did forget... This seems good to have in 0.98. I'll see what it takes to do the hadoop1 metrics. > SplitTransaction: Add metrics > - > > Key: HBASE-12779 > URL: https://issues.apache.org/jira/browse/HBASE-12779 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl > Fix For: 2.0.0, 0.98.10, 1.0.1, 0.94.27 > > Attachments: 12779.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12778) HBase
[ https://issues.apache.org/jira/browse/HBASE-12778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260762#comment-14260762 ] Lars Hofhansl commented on HBASE-12778: --- Hi [~mokai87], if you have a question, please post to the mailing lists (http://hbase.apache.org/mail-lists.html) > HBase > - > > Key: HBASE-12778 > URL: https://issues.apache.org/jira/browse/HBASE-12778 > Project: HBase > Issue Type: Bug >Reporter: mokai > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12768) Support enable cache_data_on_write in Shell while creating table
[ https://issues.apache.org/jira/browse/HBASE-12768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-12768: --- Fix Version/s: 0.98.10 2.0.0 1.0.0 Pushed to trunk. [~enis] and [~apurtell] Do you want this in branch-1 and 0.98? > Support enable cache_data_on_write in Shell while creating table > > > Key: HBASE-12768 > URL: https://issues.apache.org/jira/browse/HBASE-12768 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 1.0.0, 2.0.0, 0.94.27 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 1.0.0, 2.0.0, 0.98.10 > > Attachments: HBASE-12768.patch > > > A simple approach to support cache_data_on_write while creating table in > shell. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12768) Support enable cache_data_on_write in Shell while creating table
[ https://issues.apache.org/jira/browse/HBASE-12768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260756#comment-14260756 ] ramkrishna.s.vasudevan commented on HBASE-12768: Thanks for the reviews. Verified it in a cluster and it works fine. > Support enable cache_data_on_write in Shell while creating table > > > Key: HBASE-12768 > URL: https://issues.apache.org/jira/browse/HBASE-12768 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 1.0.0, 2.0.0, 0.94.27 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Attachments: HBASE-12768.patch > > > A simple approach to support cache_data_on_write while creating table in > shell. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12767) Fix a StoreFileScanner NPE in reverse scan flow
[
https://issues.apache.org/jira/browse/HBASE-12767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260725#comment-14260725
]
Liang Xie commented on HBASE-12767:
---
Thanks all for review, will commit shortly
> Fix a StoreFileScanner NPE in reverse scan flow
> ---
>
> Key: HBASE-12767
> URL: https://issues.apache.org/jira/browse/HBASE-12767
> Project: HBase
> Issue Type: Bug
> Components: regionserver, Scanners
>Affects Versions: 1.0.0, 2.0.0, 0.98.9
>Reporter: Liang Xie
>Assignee: Liang Xie
> Attachments: HBASE-12767.txt
>
>
> we observed a NPE in our internal 0.94 branch prd env:
> ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekToPreviousRow(StoreFileScanner.java:409)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.backwardSeek(StoreFileScanner.java:467)
> at
> org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.seekScanners(ReversedStoreScanner.java:78)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:154)
> at
> org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.(ReversedStoreScanner.java:49)
> at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2216)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:3984)
> at
> org.apache.hadoop.hbase.regionserver.ReversedRegionScannerImpl.(ReversedRegionScannerImpl.java:43)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1851)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1837)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1812)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2752)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2715)
> After tracing, it turned out a boundary bug at HalfStoreFileReader. If we
> have a reverse scan request at "bottom" part, once the start row >= the last
> row at bottom part, the NPE will be encourtered, since we will invoke
> "next()", and "atEnd = true" will be called in HalfStoreFileReader, then when
> we want to do "seekBefore" or other sth in StoreFileScanner,
> StoreFileScanner.cur will call:
> {code}
> public Cell getKeyValue() {
> if (atEnd) return null;
> {code}
> The fix is simple, we need to reset the "atEnd" flag once seekBefore op is
> successful. You can undo my HalfStoreFileReader change and run the new added
> case to see NPE.
> ps: I also found some cases in TestHRegion missing the "@Test" annotation, so
> fix them in the same patch as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12779) SplitTransaction: Add metrics
[ https://issues.apache.org/jira/browse/HBASE-12779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260710#comment-14260710 ] Andrew Purtell commented on HBASE-12779: lgtm If checking in to 0.98 don't forget hadoop1 metrics (we sometimes do). > SplitTransaction: Add metrics > - > > Key: HBASE-12779 > URL: https://issues.apache.org/jira/browse/HBASE-12779 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl > Fix For: 2.0.0, 0.98.10, 1.0.1, 0.94.27 > > Attachments: 12779.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12767) Fix a StoreFileScanner NPE in reverse scan flow
[
https://issues.apache.org/jira/browse/HBASE-12767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260707#comment-14260707
]
Anoop Sam John commented on HBASE-12767:
Looks good.
> Fix a StoreFileScanner NPE in reverse scan flow
> ---
>
> Key: HBASE-12767
> URL: https://issues.apache.org/jira/browse/HBASE-12767
> Project: HBase
> Issue Type: Bug
> Components: regionserver, Scanners
>Affects Versions: 1.0.0, 2.0.0, 0.98.9
>Reporter: Liang Xie
>Assignee: Liang Xie
> Attachments: HBASE-12767.txt
>
>
> we observed a NPE in our internal 0.94 branch prd env:
> ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekToPreviousRow(StoreFileScanner.java:409)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.backwardSeek(StoreFileScanner.java:467)
> at
> org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.seekScanners(ReversedStoreScanner.java:78)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:154)
> at
> org.apache.hadoop.hbase.regionserver.ReversedStoreScanner.(ReversedStoreScanner.java:49)
> at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2216)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:3984)
> at
> org.apache.hadoop.hbase.regionserver.ReversedRegionScannerImpl.(ReversedRegionScannerImpl.java:43)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1851)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1837)
> at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1812)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2752)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2715)
> After tracing, it turned out a boundary bug at HalfStoreFileReader. If we
> have a reverse scan request at "bottom" part, once the start row >= the last
> row at bottom part, the NPE will be encourtered, since we will invoke
> "next()", and "atEnd = true" will be called in HalfStoreFileReader, then when
> we want to do "seekBefore" or other sth in StoreFileScanner,
> StoreFileScanner.cur will call:
> {code}
> public Cell getKeyValue() {
> if (atEnd) return null;
> {code}
> The fix is simple, we need to reset the "atEnd" flag once seekBefore op is
> successful. You can undo my HalfStoreFileReader change and run the new added
> case to see NPE.
> ps: I also found some cases in TestHRegion missing the "@Test" annotation, so
> fix them in the same patch as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12772) TestPerColumnFamilyFlush failing
[
https://issues.apache.org/jira/browse/HBASE-12772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260692#comment-14260692
]
Dima Spivak commented on HBASE-12772:
-
Yessir! [Here's the
TEST-org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.xml|https://gist.github.com/dimaspivak/d28e109ac52e4cc327f3].
Let me know if this is missing anything.
> TestPerColumnFamilyFlush failing
>
>
> Key: HBASE-12772
> URL: https://issues.apache.org/jira/browse/HBASE-12772
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.0.0
>Reporter: stack
> Attachments: 0001-HBASE-12772-TestPerColumnFamilyFlush-failing.patch
>
>
> On internal rig see this failing in two places:
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling
> Failing for the past 1 build (Since Failed#653 )
> Took 9 sec.
> Error Message
> expected:<424> but was:<205744>
> Stacktrace
> java.lang.AssertionError: expected:<424> but was:<205744>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling(TestPerColumnFamilyFlush.java:483)
> {code}
> and
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay
> Failing for the past 1 build (Since Failed#653 )
> Took 25 ms.
> Error Message
> A mini-cluster is already running
> Stacktrace
> java.lang.IllegalStateException: A mini-cluster is already running
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:921)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:812)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:794)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:781)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplay(TestPerColumnFamilyFlush.java:337)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay(TestPerColumnFamilyFlush.java:418)
> {code}
> Opening issue to keep an eye on these tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12772) TestPerColumnFamilyFlush failing
[
https://issues.apache.org/jira/browse/HBASE-12772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260690#comment-14260690
]
stack commented on HBASE-12772:
---
@dima spivak u have a failing log?,,,so can see if same failure or new one.
Will try later.this evening
> TestPerColumnFamilyFlush failing
>
>
> Key: HBASE-12772
> URL: https://issues.apache.org/jira/browse/HBASE-12772
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.0.0
>Reporter: stack
> Attachments: 0001-HBASE-12772-TestPerColumnFamilyFlush-failing.patch
>
>
> On internal rig see this failing in two places:
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling
> Failing for the past 1 build (Since Failed#653 )
> Took 9 sec.
> Error Message
> expected:<424> but was:<205744>
> Stacktrace
> java.lang.AssertionError: expected:<424> but was:<205744>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling(TestPerColumnFamilyFlush.java:483)
> {code}
> and
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay
> Failing for the past 1 build (Since Failed#653 )
> Took 25 ms.
> Error Message
> A mini-cluster is already running
> Stacktrace
> java.lang.IllegalStateException: A mini-cluster is already running
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:921)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:812)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:794)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:781)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplay(TestPerColumnFamilyFlush.java:337)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay(TestPerColumnFamilyFlush.java:418)
> {code}
> Opening issue to keep an eye on these tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12775) CompressionTest ate my HFile (sigh!)
[
https://issues.apache.org/jira/browse/HBASE-12775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260687#comment-14260687
]
Dima Spivak commented on HBASE-12775:
-
It's probably best to use the Log instance rather than System.err.
> CompressionTest ate my HFile (sigh!)
>
>
> Key: HBASE-12775
> URL: https://issues.apache.org/jira/browse/HBASE-12775
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 0.98.9, 0.99.2
>Reporter: Aditya Kishore
>Assignee: Aditya Kishore
> Fix For: 2.0.0
>
> Attachments: HBASE-12775-CompressionTest-ate-my-HFile.patch
>
>
> {{org.apache.hadoop.hbase.util.CompressionTest}} should abort execution if
> the file specified on the command line exists. This will help careless (me)
> or unsuspecting user to not loose data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-9117) Remove HTablePool and all HConnection pooling related APIs
[ https://issues.apache.org/jira/browse/HBASE-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov reassigned HBASE-9117: -- Assignee: Mikhail Antonov (was: Nick Dimiduk) > Remove HTablePool and all HConnection pooling related APIs > -- > > Key: HBASE-9117 > URL: https://issues.apache.org/jira/browse/HBASE-9117 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Mikhail Antonov >Priority: Critical > Fix For: 2.0.0, 0.99.2 > > Attachments: HBASE-9117.00.patch, HBASE-9117.01.patch, > HBASE-9117.02.patch, HBASE-9117.03.patch, HBASE-9117.04.patch, > HBASE-9117.05.patch, HBASE-9117.06.patch > > > The recommended way is now: > # Create an HConnection: HConnectionManager.createConnection(...) > # Create a light HTable: HConnection.getTable(...) > # table.close() > # connection.close() > All other API and pooling will be removed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12768) Support enable cache_data_on_write in Shell while creating table
[ https://issues.apache.org/jira/browse/HBASE-12768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260658#comment-14260658 ] Anoop Sam John commented on HBASE-12768: lgtm > Support enable cache_data_on_write in Shell while creating table > > > Key: HBASE-12768 > URL: https://issues.apache.org/jira/browse/HBASE-12768 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 1.0.0, 2.0.0, 0.94.27 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Attachments: HBASE-12768.patch > > > A simple approach to support cache_data_on_write while creating table in > shell. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-12780) HBase
[ https://issues.apache.org/jira/browse/HBASE-12780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved HBASE-12780. Resolution: Invalid > HBase > - > > Key: HBASE-12780 > URL: https://issues.apache.org/jira/browse/HBASE-12780 > Project: HBase > Issue Type: Bug >Reporter: mokai > > Invalid issue, please ignore it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12774) Fix the inconsistent permission checks for bulkloading.
[
https://issues.apache.org/jira/browse/HBASE-12774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260648#comment-14260648
]
Hadoop QA commented on HBASE-12774:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12689396/HBASE-12774.patch
against master branch at commit b2eea8cac6cceab323bf79b77e321d24dd5c90c2.
ATTACHMENT ID: 12689396
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/12253//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12253//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12253//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12253//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12253//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12253//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12253//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12253//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12253//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12253//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12253//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12253//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/12253//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12253//console
This message is automatically generated.
> Fix the inconsistent permission checks for bulkloading.
> ---
>
> Key: HBASE-12774
> URL: https://issues.apache.org/jira/browse/HBASE-12774
> Project: HBase
> Issue Type: Improvement
>Reporter: Srikanth Srungarapu
>Assignee: Srikanth Srungarapu
>Priority: Minor
> Attachments: HBASE-12774.patch
>
>
> Three checks(prePrepareBulkLoad, preCleanupBulkLoad, preBulkLoadHFile) are
> being done while performing secure bulk load, and it looks the former two
> checks for 'W', while the later checks for 'C'. After having offline chat
> with [~jdcryans], looks like the inconsistency among checks were unintended.
> So, we can address this multiple ways.
> * All checks should be for 'W'
> * All checks should be for 'C'
> * All checks should be for both 'W' and 'C'
> Posting the initial patch going by the first option. Open to discussion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12775) CompressionTest ate my HFile (sigh!)
[
https://issues.apache.org/jira/browse/HBASE-12775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260638#comment-14260638
]
Hadoop QA commented on HBASE-12775:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12689397/HBASE-12775-CompressionTest-ate-my-HFile.patch
against master branch at commit b2eea8cac6cceab323bf79b77e321d24dd5c90c2.
ATTACHMENT ID: 12689397
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.coprocessor.TestMasterObserver.testRegionTransitionOperations(TestMasterObserver.java:1605)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/12252//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12252//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12252//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12252//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12252//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12252//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12252//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12252//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12252//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12252//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12252//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12252//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/12252//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12252//console
This message is automatically generated.
> CompressionTest ate my HFile (sigh!)
>
>
> Key: HBASE-12775
> URL: https://issues.apache.org/jira/browse/HBASE-12775
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 0.98.9, 0.99.2
>Reporter: Aditya Kishore
>Assignee: Aditya Kishore
> Fix For: 2.0.0
>
> Attachments: HBASE-12775-CompressionTest-ate-my-HFile.patch
>
>
> {{org.apache.hadoop.hbase.util.CompressionTest}} should abort execution if
> the file specified on the command line exists. This will help careless (me)
> or unsuspecting user to not loose data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12772) TestPerColumnFamilyFlush failing
[
https://issues.apache.org/jira/browse/HBASE-12772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260639#comment-14260639
]
Hudson commented on HBASE-12772:
FAILURE: Integrated in HBase-TRUNK #5973 (See
[https://builds.apache.org/job/HBase-TRUNK/5973/])
HBASE-12772 TestPerColumnFamilyFlush failing (stack: rev
02b03326506d5ffeb191a61a598763a99380e2a0)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestPerColumnFamilyFlush.java
> TestPerColumnFamilyFlush failing
>
>
> Key: HBASE-12772
> URL: https://issues.apache.org/jira/browse/HBASE-12772
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.0.0
>Reporter: stack
> Attachments: 0001-HBASE-12772-TestPerColumnFamilyFlush-failing.patch
>
>
> On internal rig see this failing in two places:
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling
> Failing for the past 1 build (Since Failed#653 )
> Took 9 sec.
> Error Message
> expected:<424> but was:<205744>
> Stacktrace
> java.lang.AssertionError: expected:<424> but was:<205744>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling(TestPerColumnFamilyFlush.java:483)
> {code}
> and
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay
> Failing for the past 1 build (Since Failed#653 )
> Took 25 ms.
> Error Message
> A mini-cluster is already running
> Stacktrace
> java.lang.IllegalStateException: A mini-cluster is already running
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:921)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:812)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:794)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:781)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplay(TestPerColumnFamilyFlush.java:337)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay(TestPerColumnFamilyFlush.java:418)
> {code}
> Opening issue to keep an eye on these tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12028) Abort the RegionServer, when one of it's handler threads die
[
https://issues.apache.org/jira/browse/HBASE-12028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260636#comment-14260636
]
Hadoop QA commented on HBASE-12028:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12689395/hbase-12028-v5.patch
against master branch at commit b2eea8cac6cceab323bf79b77e321d24dd5c90c2.
ATTACHMENT ID: 12689395
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
2086 checkstyle errors (more than the master's current 2081 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/12251//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12251//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12251//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12251//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12251//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12251//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12251//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12251//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12251//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12251//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12251//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12251//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/12251//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12251//console
This message is automatically generated.
> Abort the RegionServer, when one of it's handler threads die
>
>
> Key: HBASE-12028
> URL: https://issues.apache.org/jira/browse/HBASE-12028
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Sudarshan Kadambi
>Assignee: Alicia Ying Shu
> Attachments: Hbase-12028-v3.patch, Hbase-12028.patch,
> hbase-12028-v4.patch, hbase-12028-v5.patch
>
>
> Over in HBase-11813, a user identified an issue where in all the RPC handler
> threads would exit with StackOverflow errors due to an unchecked
> recursion-terminating condition. Our clusters demonstrated the same trace.
> While the patch posted for HBASE-11813 got our clusters to be merry again,
> the breakdown surfaced some larger issues.
> When the RegionServer had all it's RPC handler threads dead, it continued to
> have regions assigned it. Clearly, it wouldn't be able to serve reads and
> writes on those regions. A second issue was that when a user tried to disable
> or drop a table, the master would try to communicate to the regionserver for
> region unassignment. Since the same handler threads seem to be used for
> master <-> RS communication as well, the master ended up hanging on the RS
> indefinitely. Eventually, the mas
[jira] [Commented] (HBASE-12776) SpliTransaction: Log number of files to be split
[ https://issues.apache.org/jira/browse/HBASE-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260634#comment-14260634 ] Lars Hofhansl commented on HBASE-12776: --- My first patch had this at INFO, but then I wasn't sure. INFO seems better, indeed. Will do on commit. > SpliTransaction: Log number of files to be split > > > Key: HBASE-12776 > URL: https://issues.apache.org/jira/browse/HBASE-12776 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Priority: Minor > Fix For: 2.0.0, 0.98.10, 1.0.1, 0.94.27 > > Attachments: 12765.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12779) SplitTransaction: Add metrics
[ https://issues.apache.org/jira/browse/HBASE-12779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-12779: -- Fix Version/s: 0.94.27 1.0.1 0.98.10 2.0.0 > SplitTransaction: Add metrics > - > > Key: HBASE-12779 > URL: https://issues.apache.org/jira/browse/HBASE-12779 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl > Fix For: 2.0.0, 0.98.10, 1.0.1, 0.94.27 > > Attachments: 12779.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12779) SplitTransaction: Add metrics
[ https://issues.apache.org/jira/browse/HBASE-12779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-12779: -- Attachment: 12779.txt Initial patch. Completely untested, only know that it compiles. Parking it here. This was easy since it all fit into SpliRequest. I also want to add the number of files splits and the number of files for daughter A and daughter B. That would have to caught in SplitTransaction. [~eclark], FYI. > SplitTransaction: Add metrics > - > > Key: HBASE-12779 > URL: https://issues.apache.org/jira/browse/HBASE-12779 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl > Fix For: 2.0.0, 0.98.10, 1.0.1, 0.94.27 > > Attachments: 12779.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12772) TestPerColumnFamilyFlush failing
[
https://issues.apache.org/jira/browse/HBASE-12772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260628#comment-14260628
]
Dima Spivak commented on HBASE-12772:
-
:-\ Looks to still be failing occasionally. I ran the test in a loop with your
latest commit, [~stack], and got this after the 4th run:
{code}
Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 74.859 sec <<<
FAILURE! - in org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush
testFlushingWhenLogRolling(org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush)
Time elapsed: 11.267 sec <<< FAILURE!
java.lang.AssertionError: expected:<424> but was:<205392>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling(TestPerColumnFamilyFlush.java:479)
{code}
which looks like what it was throwing before. [~Apache9], what do you suggest
take place of the 4 s sleep in testFlushingWhenLogRolling?
> TestPerColumnFamilyFlush failing
>
>
> Key: HBASE-12772
> URL: https://issues.apache.org/jira/browse/HBASE-12772
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.0.0
>Reporter: stack
> Attachments: 0001-HBASE-12772-TestPerColumnFamilyFlush-failing.patch
>
>
> On internal rig see this failing in two places:
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling
> Failing for the past 1 build (Since Failed#653 )
> Took 9 sec.
> Error Message
> expected:<424> but was:<205744>
> Stacktrace
> java.lang.AssertionError: expected:<424> but was:<205744>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling(TestPerColumnFamilyFlush.java:483)
> {code}
> and
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay
> Failing for the past 1 build (Since Failed#653 )
> Took 25 ms.
> Error Message
> A mini-cluster is already running
> Stacktrace
> java.lang.IllegalStateException: A mini-cluster is already running
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:921)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:812)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:794)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:781)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplay(TestPerColumnFamilyFlush.java:337)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay(TestPerColumnFamilyFlush.java:418)
> {code}
> Opening issue to keep an eye on these tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12780) HBase
[ https://issues.apache.org/jira/browse/HBASE-12780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mokai updated HBASE-12780: -- Description: Invalid issue, please ignore it. > HBase > - > > Key: HBASE-12780 > URL: https://issues.apache.org/jira/browse/HBASE-12780 > Project: HBase > Issue Type: Bug >Reporter: mokai > > Invalid issue, please ignore it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-12778) HBase
[ https://issues.apache.org/jira/browse/HBASE-12778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-12778. --- Resolution: Invalid Nothing in this resolving as invalid > HBase > - > > Key: HBASE-12778 > URL: https://issues.apache.org/jira/browse/HBASE-12778 > Project: HBase > Issue Type: Bug >Reporter: mokai > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12772) TestPerColumnFamilyFlush failing
[
https://issues.apache.org/jira/browse/HBASE-12772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260622#comment-14260622
]
Hudson commented on HBASE-12772:
FAILURE: Integrated in HBase-1.1 #32 (See
[https://builds.apache.org/job/HBase-1.1/32/])
HBASE-12772 TestPerColumnFamilyFlush failing (stack: rev
8ff62d9ceed6f1a374ad12ce080eddb240f3f9dd)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestPerColumnFamilyFlush.java
> TestPerColumnFamilyFlush failing
>
>
> Key: HBASE-12772
> URL: https://issues.apache.org/jira/browse/HBASE-12772
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.0.0
>Reporter: stack
> Attachments: 0001-HBASE-12772-TestPerColumnFamilyFlush-failing.patch
>
>
> On internal rig see this failing in two places:
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling
> Failing for the past 1 build (Since Failed#653 )
> Took 9 sec.
> Error Message
> expected:<424> but was:<205744>
> Stacktrace
> java.lang.AssertionError: expected:<424> but was:<205744>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling(TestPerColumnFamilyFlush.java:483)
> {code}
> and
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay
> Failing for the past 1 build (Since Failed#653 )
> Took 25 ms.
> Error Message
> A mini-cluster is already running
> Stacktrace
> java.lang.IllegalStateException: A mini-cluster is already running
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:921)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:812)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:794)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:781)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplay(TestPerColumnFamilyFlush.java:337)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay(TestPerColumnFamilyFlush.java:418)
> {code}
> Opening issue to keep an eye on these tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12779) SplitTransaction: Add metrics
Lars Hofhansl created HBASE-12779: - Summary: SplitTransaction: Add metrics Key: HBASE-12779 URL: https://issues.apache.org/jira/browse/HBASE-12779 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12780) HBase
mokai created HBASE-12780: - Summary: HBase Key: HBASE-12780 URL: https://issues.apache.org/jira/browse/HBASE-12780 Project: HBase Issue Type: Bug Reporter: mokai -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12778) HBase
mokai created HBASE-12778: - Summary: HBase Key: HBASE-12778 URL: https://issues.apache.org/jira/browse/HBASE-12778 Project: HBase Issue Type: Bug Reporter: mokai -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12772) TestPerColumnFamilyFlush failing
[
https://issues.apache.org/jira/browse/HBASE-12772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260621#comment-14260621
]
stack commented on HBASE-12772:
---
[~Apache9] I think the 'fix' here might be enough after all (I was looking at
old test results). The test should do as you intended. Go back to your
year-end paperwork!
> TestPerColumnFamilyFlush failing
>
>
> Key: HBASE-12772
> URL: https://issues.apache.org/jira/browse/HBASE-12772
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.0.0
>Reporter: stack
> Attachments: 0001-HBASE-12772-TestPerColumnFamilyFlush-failing.patch
>
>
> On internal rig see this failing in two places:
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling
> Failing for the past 1 build (Since Failed#653 )
> Took 9 sec.
> Error Message
> expected:<424> but was:<205744>
> Stacktrace
> java.lang.AssertionError: expected:<424> but was:<205744>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling(TestPerColumnFamilyFlush.java:483)
> {code}
> and
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay
> Failing for the past 1 build (Since Failed#653 )
> Took 25 ms.
> Error Message
> A mini-cluster is already running
> Stacktrace
> java.lang.IllegalStateException: A mini-cluster is already running
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:921)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:812)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:794)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:781)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplay(TestPerColumnFamilyFlush.java:337)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay(TestPerColumnFamilyFlush.java:418)
> {code}
> Opening issue to keep an eye on these tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12773) Add warning message when user is trying to bulkload a large HFile.
[
https://issues.apache.org/jira/browse/HBASE-12773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260617#comment-14260617
]
Hadoop QA commented on HBASE-12773:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12689392/HBASE-12773.patch
against master branch at commit b2eea8cac6cceab323bf79b77e321d24dd5c90c2.
ATTACHMENT ID: 12689392
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/12250//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12250//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12250//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12250//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12250//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12250//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12250//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12250//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12250//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12250//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12250//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12250//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/12250//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12250//console
This message is automatically generated.
> Add warning message when user is trying to bulkload a large HFile.
> --
>
> Key: HBASE-12773
> URL: https://issues.apache.org/jira/browse/HBASE-12773
> Project: HBase
> Issue Type: Improvement
>Reporter: Srikanth Srungarapu
>Assignee: Srikanth Srungarapu
>Priority: Minor
> Attachments: HBASE-12773.patch
>
>
> As per the description.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12772) TestPerColumnFamilyFlush failing
[
https://issues.apache.org/jira/browse/HBASE-12772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260615#comment-14260615
]
zhangduo commented on HBASE-12772:
--
I think the problem is the test only use a Thread.sleep but do not check
whether the event actual happened.
Let me see if we have a better choice rather than Thread.sleep.
Thanks, [~stack]
> TestPerColumnFamilyFlush failing
>
>
> Key: HBASE-12772
> URL: https://issues.apache.org/jira/browse/HBASE-12772
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.0.0
>Reporter: stack
> Attachments: 0001-HBASE-12772-TestPerColumnFamilyFlush-failing.patch
>
>
> On internal rig see this failing in two places:
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling
> Failing for the past 1 build (Since Failed#653 )
> Took 9 sec.
> Error Message
> expected:<424> but was:<205744>
> Stacktrace
> java.lang.AssertionError: expected:<424> but was:<205744>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling(TestPerColumnFamilyFlush.java:483)
> {code}
> and
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay
> Failing for the past 1 build (Since Failed#653 )
> Took 25 ms.
> Error Message
> A mini-cluster is already running
> Stacktrace
> java.lang.IllegalStateException: A mini-cluster is already running
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:921)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:812)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:794)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:781)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplay(TestPerColumnFamilyFlush.java:337)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay(TestPerColumnFamilyFlush.java:418)
> {code}
> Opening issue to keep an eye on these tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12776) SpliTransaction: Log number of files to be split
[ https://issues.apache.org/jira/browse/HBASE-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260593#comment-14260593 ] Andrew Purtell commented on HBASE-12776: Also consider logging the first line at INFO. Either way, it's fine. > SpliTransaction: Log number of files to be split > > > Key: HBASE-12776 > URL: https://issues.apache.org/jira/browse/HBASE-12776 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Priority: Minor > Fix For: 2.0.0, 0.98.10, 1.0.1, 0.94.27 > > Attachments: 12765.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12776) SpliTransaction: Log number of files to be split
[ https://issues.apache.org/jira/browse/HBASE-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260591#comment-14260591 ] Andrew Purtell commented on HBASE-12776: +1 > SpliTransaction: Log number of files to be split > > > Key: HBASE-12776 > URL: https://issues.apache.org/jira/browse/HBASE-12776 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Priority: Minor > Fix For: 2.0.0, 0.98.10, 1.0.1, 0.94.27 > > Attachments: 12765.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12769) Replication fails to delete all corresponding zk nodes when peer is removed
[ https://issues.apache.org/jira/browse/HBASE-12769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260589#comment-14260589 ] Andrew Purtell commented on HBASE-12769: Agreed about HBASE-11392 but gating what sounds like a fix for a production pain on development of a big new coordination framework we otherwise haven't found the time to do isn't fair to the reporter IMHO > Replication fails to delete all corresponding zk nodes when peer is removed > --- > > Key: HBASE-12769 > URL: https://issues.apache.org/jira/browse/HBASE-12769 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 0.99.2 >Reporter: cuijianwei >Priority: Minor > > When removing a peer, the client side will delete peerId under peersZNode > node; then alive region servers will be notified and delete corresponding > hlog queues under its rsZNode of replication. However, if there are failed > servers whose hlog queues have not been transferred by alive servers(this > likely happens if setting a big value to "replication.sleep.before.failover" > and lots of region servers restarted), these hlog queues won't be deleted > after the peer is removed. I think remove_peer should guarantee all > corresponding zk nodes have been removed after it completes; otherwise, if we > create a new peer with the same peerId with the removed one, there might be > unexpected data to be replicated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12772) TestPerColumnFamilyFlush failing
[
https://issues.apache.org/jira/browse/HBASE-12772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260583#comment-14260583
]
stack commented on HBASE-12772:
---
Might not be enough... added logging though. Lets see.
> TestPerColumnFamilyFlush failing
>
>
> Key: HBASE-12772
> URL: https://issues.apache.org/jira/browse/HBASE-12772
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.0.0
>Reporter: stack
> Attachments: 0001-HBASE-12772-TestPerColumnFamilyFlush-failing.patch
>
>
> On internal rig see this failing in two places:
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling
> Failing for the past 1 build (Since Failed#653 )
> Took 9 sec.
> Error Message
> expected:<424> but was:<205744>
> Stacktrace
> java.lang.AssertionError: expected:<424> but was:<205744>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling(TestPerColumnFamilyFlush.java:483)
> {code}
> and
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay
> Failing for the past 1 build (Since Failed#653 )
> Took 25 ms.
> Error Message
> A mini-cluster is already running
> Stacktrace
> java.lang.IllegalStateException: A mini-cluster is already running
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:921)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:812)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:794)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:781)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplay(TestPerColumnFamilyFlush.java:337)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay(TestPerColumnFamilyFlush.java:418)
> {code}
> Opening issue to keep an eye on these tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12777) Multi-page book has broken links that work in the single-page version
Dima Spivak created HBASE-12777:
---
Summary: Multi-page book has broken links that work in the
single-page version
Key: HBASE-12777
URL: https://issues.apache.org/jira/browse/HBASE-12777
Project: HBase
Issue Type: Bug
Components: documentation
Reporter: Dima Spivak
Assignee: Misty Stanley-Jones
Noticed this today in a few places. As an example, on [this page of the
multi-page book|http://hbase.apache.org/book/regions.arch.html], all the links
are broken (because their {{href}} attribute is rendered as empty). Meanwhile,
the links are present and functional [in the single-page
book|http://hbase.apache.org/book.html#regions.arch].
Do you happen to know why this is happening, [~misty]? I'd be happy to help you
investigate, but I imagine you already know what's going on at a glance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12765) SplitTransaction creates too many threads (potentially)
[ https://issues.apache.org/jira/browse/HBASE-12765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260571#comment-14260571 ] Elliott Clark commented on HBASE-12765: --- I would think making a counter off of MetricsRegionServerSource would be good. I usually add the counter to HRegionServer if that number will be used somewhere else ( server load reporting, logging, etc) otherwise I add it to MetricsRegionServerSource. I can do that work if you want. > SplitTransaction creates too many threads (potentially) > --- > > Key: HBASE-12765 > URL: https://issues.apache.org/jira/browse/HBASE-12765 > Project: HBase > Issue Type: Brainstorming >Reporter: Lars Hofhansl > Attachments: 12765.txt > > > In splitStoreFiles(...) we create a new thread pool with as many threads as > there are files to split. > We should be able to do better. During times of very heavy write loads there > might be a lot of files to split and multiple splits might be going on at the > same time on the same region server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12774) Fix the inconsistent permission checks for bulkloading.
[ https://issues.apache.org/jira/browse/HBASE-12774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260570#comment-14260570 ] Jean-Daniel Cryans commented on HBASE-12774: Like we discussed, and because of HBASE-11008, it needs to be C which is your second option. Now, adding a C+W check isn't something we're doing anywhere else (I think?) and I'm not sure if it makes sense. The way things currently work, I'm not sure if it makes sense for a user to have C without having W. > Fix the inconsistent permission checks for bulkloading. > --- > > Key: HBASE-12774 > URL: https://issues.apache.org/jira/browse/HBASE-12774 > Project: HBase > Issue Type: Improvement >Reporter: Srikanth Srungarapu >Assignee: Srikanth Srungarapu >Priority: Minor > Attachments: HBASE-12774.patch > > > Three checks(prePrepareBulkLoad, preCleanupBulkLoad, preBulkLoadHFile) are > being done while performing secure bulk load, and it looks the former two > checks for 'W', while the later checks for 'C'. After having offline chat > with [~jdcryans], looks like the inconsistency among checks were unintended. > So, we can address this multiple ways. > * All checks should be for 'W' > * All checks should be for 'C' > * All checks should be for both 'W' and 'C' > Posting the initial patch going by the first option. Open to discussion. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12765) SplitTransaction creates too many threads (potentially)
[ https://issues.apache.org/jira/browse/HBASE-12765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260566#comment-14260566 ] Lars Hofhansl commented on HBASE-12765: --- [~eclark], do you have a recommendation as to where the keep track of split metrics? It does not make sense to put then on the region as the parent region will go away, this would need to be stored per region server somehow. I could add counters to HRegionServer or maybe add new counters to MetricsRegionServerSource? > SplitTransaction creates too many threads (potentially) > --- > > Key: HBASE-12765 > URL: https://issues.apache.org/jira/browse/HBASE-12765 > Project: HBase > Issue Type: Brainstorming >Reporter: Lars Hofhansl > Attachments: 12765.txt > > > In splitStoreFiles(...) we create a new thread pool with as many threads as > there are files to split. > We should be able to do better. During times of very heavy write loads there > might be a lot of files to split and multiple splits might be going on at the > same time on the same region server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12776) SpliTransaction: Log number of files to be split
[ https://issues.apache.org/jira/browse/HBASE-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260560#comment-14260560 ] Lars Hofhansl commented on HBASE-12776: --- Stack +1's on parent, just adds two debug log statements. Going to commit to all branches. This will be valuable to decide how to size the thread pool for splits in the future. > SpliTransaction: Log number of files to be split > > > Key: HBASE-12776 > URL: https://issues.apache.org/jira/browse/HBASE-12776 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Priority: Minor > Fix For: 2.0.0, 0.98.10, 1.0.1, 0.94.27 > > Attachments: 12765.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12772) TestPerColumnFamilyFlush failing
[
https://issues.apache.org/jira/browse/HBASE-12772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260562#comment-14260562
]
stack commented on HBASE-12772:
---
I pushed to branch-1+
This test fails frequently. Lets see if this commit fixes it. Will wait on
test runs to close. Test fails frequently internally.
> TestPerColumnFamilyFlush failing
>
>
> Key: HBASE-12772
> URL: https://issues.apache.org/jira/browse/HBASE-12772
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.0.0
>Reporter: stack
> Attachments: 0001-HBASE-12772-TestPerColumnFamilyFlush-failing.patch
>
>
> On internal rig see this failing in two places:
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling
> Failing for the past 1 build (Since Failed#653 )
> Took 9 sec.
> Error Message
> expected:<424> but was:<205744>
> Stacktrace
> java.lang.AssertionError: expected:<424> but was:<205744>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling(TestPerColumnFamilyFlush.java:483)
> {code}
> and
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay
> Failing for the past 1 build (Since Failed#653 )
> Took 25 ms.
> Error Message
> A mini-cluster is already running
> Stacktrace
> java.lang.IllegalStateException: A mini-cluster is already running
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:921)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:812)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:794)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:781)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplay(TestPerColumnFamilyFlush.java:337)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay(TestPerColumnFamilyFlush.java:418)
> {code}
> Opening issue to keep an eye on these tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12776) SpliTransaction: Log number of files to be split
[ https://issues.apache.org/jira/browse/HBASE-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-12776: -- Fix Version/s: (was: 1.0.0) 1.0.1 > SpliTransaction: Log number of files to be split > > > Key: HBASE-12776 > URL: https://issues.apache.org/jira/browse/HBASE-12776 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Priority: Minor > Fix For: 2.0.0, 0.98.10, 1.0.1, 0.94.27 > > Attachments: 12765.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12776) SpliTransaction: Log number of files to be split
[ https://issues.apache.org/jira/browse/HBASE-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-12776: -- Attachment: 12765.txt Trivial patch from parent. > SpliTransaction: Log number of files to be split > > > Key: HBASE-12776 > URL: https://issues.apache.org/jira/browse/HBASE-12776 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Priority: Minor > Fix For: 2.0.0, 0.98.10, 1.0.1, 0.94.27 > > Attachments: 12765.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12776) SpliTransaction: Log number of files to be split
Lars Hofhansl created HBASE-12776: - Summary: SpliTransaction: Log number of files to be split Key: HBASE-12776 URL: https://issues.apache.org/jira/browse/HBASE-12776 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Priority: Minor Fix For: 1.0.0, 2.0.0, 0.98.10, 0.94.27 Attachments: 12765.txt -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12772) TestPerColumnFamilyFlush failing
[
https://issues.apache.org/jira/browse/HBASE-12772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-12772:
--
Attachment: 0001-HBASE-12772-TestPerColumnFamilyFlush-failing.patch
Patch I'm going to commit.
> TestPerColumnFamilyFlush failing
>
>
> Key: HBASE-12772
> URL: https://issues.apache.org/jira/browse/HBASE-12772
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.0.0
>Reporter: stack
> Attachments: 0001-HBASE-12772-TestPerColumnFamilyFlush-failing.patch
>
>
> On internal rig see this failing in two places:
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling
> Failing for the past 1 build (Since Failed#653 )
> Took 9 sec.
> Error Message
> expected:<424> but was:<205744>
> Stacktrace
> java.lang.AssertionError: expected:<424> but was:<205744>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling(TestPerColumnFamilyFlush.java:483)
> {code}
> and
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay
> Failing for the past 1 build (Since Failed#653 )
> Took 25 ms.
> Error Message
> A mini-cluster is already running
> Stacktrace
> java.lang.IllegalStateException: A mini-cluster is already running
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:921)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:812)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:794)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:781)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplay(TestPerColumnFamilyFlush.java:337)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay(TestPerColumnFamilyFlush.java:418)
> {code}
> Opening issue to keep an eye on these tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12772) TestPerColumnFamilyFlush failing
[
https://issues.apache.org/jira/browse/HBASE-12772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260551#comment-14260551
]
stack commented on HBASE-12772:
---
Looks like test fails because we mod the namespace table. It has oldest edit.
Its region gets flushed because it is holding up the outstanding WALs. What is
wanted is that we flush the region that has all the edits in it, the ones added
by the test. I changed the test so rather than make the test table in a new
ns, instead, it just uses the default. This way, no edits in namespace table
and we should get the right region each time through.
Second test failed because already a minihbasecluster running -- it wasn't
being cleaned up by first test fail.
> TestPerColumnFamilyFlush failing
>
>
> Key: HBASE-12772
> URL: https://issues.apache.org/jira/browse/HBASE-12772
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.0.0
>Reporter: stack
>
> On internal rig see this failing in two places:
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling
> Failing for the past 1 build (Since Failed#653 )
> Took 9 sec.
> Error Message
> expected:<424> but was:<205744>
> Stacktrace
> java.lang.AssertionError: expected:<424> but was:<205744>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testFlushingWhenLogRolling(TestPerColumnFamilyFlush.java:483)
> {code}
> and
> {code}
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay
> Failing for the past 1 build (Since Failed#653 )
> Took 25 ms.
> Error Message
> A mini-cluster is already running
> Stacktrace
> java.lang.IllegalStateException: A mini-cluster is already running
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:921)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:812)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:794)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:781)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplay(TestPerColumnFamilyFlush.java:337)
> at
> org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush.testLogReplayWithDistributedReplay(TestPerColumnFamilyFlush.java:418)
> {code}
> Opening issue to keep an eye on these tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12728) buffered writes substantially less useful after removal of HTablePool
[
https://issues.apache.org/jira/browse/HBASE-12728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260547#comment-14260547
]
Lars Hofhansl commented on HBASE-12728:
---
If we do not want an API change for the existing interface we can invent a
"LightweightTable" or "UnbufferedTable". But I'd prefer BufferedTable as a new
interface.
Too late for 1.0? [~enis].
> buffered writes substantially less useful after removal of HTablePool
> -
>
> Key: HBASE-12728
> URL: https://issues.apache.org/jira/browse/HBASE-12728
> Project: HBase
> Issue Type: Bug
> Components: hbase
>Affects Versions: 0.98.0
>Reporter: Aaron Beppu
>
> In previous versions of HBase, when use of HTablePool was encouraged, HTable
> instances were long-lived in that pool, and for that reason, if autoFlush was
> set to false, the table instance could accumulate a full buffer of writes
> before a flush was triggered. Writes from the client to the cluster could
> then be substantially larger and less frequent than without buffering.
> However, when HTablePool was deprecated, the primary justification seems to
> have been that creating HTable instances is cheap, so long as the connection
> and executor service being passed to it are pre-provided. A use pattern was
> encouraged where users should create a new HTable instance for every
> operation, using an existing connection and executor service, and then close
> the table. In this pattern, buffered writes are substantially less useful;
> writes are as small and as frequent as they would have been with
> autoflush=true, except the synchronous write is moved from the operation
> itself to the table close call which immediately follows.
> More concretely :
> ```
> // Given these two helpers ...
> private HTableInterface getAutoFlushTable(String tableName) throws
> IOException {
> // (autoflush is true by default)
> return storedConnection.getTable(tableName, executorService);
> }
> private HTableInterface getBufferedTable(String tableName) throws IOException
> {
> HTableInterface table = getAutoFlushTable(tableName);
> table.setAutoFlush(false);
> return table;
> }
> // it's my contention that these two methods would behave almost identically,
> // except the first will hit a synchronous flush during the put call,
> and the second will
> // flush during the (hidden) close call on table.
> private void writeAutoFlushed(Put somePut) throws IOException {
> try (HTableInterface table = getAutoFlushTable(tableName)) {
> table.put(somePut); // will do synchronous flush
> }
> }
> private void writeBuffered(Put somePut) throws IOException {
> try (HTableInterface table = getBufferedTable(tableName)) {
> table.put(somePut);
> } // auto-close will trigger synchronous flush
> }
> ```
> For buffered writes to actually provide a performance benefit to users, one
> of two things must happen:
> - The writeBuffer itself shouldn't live, flush and die with the lifecycle of
> it's HTableInstance. If the writeBuffer were managed elsewhere and had a long
> lifespan, this could cease to be an issue. However, if the same writeBuffer
> is appended to by multiple tables, then some additional concurrency control
> will be needed around it.
> - Alternatively, there should be some pattern for having long-lived HTable
> instances. However, since HTable is not thread-safe, we'd need multiple
> instances, and a mechanism for leasing them out safely -- which sure sounds a
> lot like the old HTablePool to me.
> See discussion on mailing list here :
> http://mail-archives.apache.org/mod_mbox/hbase-user/201412.mbox/%3CCAPdJLkEzmUQZ_kvD%3D8mrxi4V%3DhCmUp3g9MUZsddD%2Bmon%2BAvNtg%40mail.gmail.com%3E
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12775) CompressionTest ate my HFile (sigh!)
[
https://issues.apache.org/jira/browse/HBASE-12775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aditya Kishore updated HBASE-12775:
---
Fix Version/s: 2.0.0
Status: Patch Available (was: Open)
Submitting the trivial patch which will exit the CompressionTest if the
specified file exists.
> CompressionTest ate my HFile (sigh!)
>
>
> Key: HBASE-12775
> URL: https://issues.apache.org/jira/browse/HBASE-12775
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 0.99.2, 0.98.9
>Reporter: Aditya Kishore
>Assignee: Aditya Kishore
> Fix For: 2.0.0
>
> Attachments: HBASE-12775-CompressionTest-ate-my-HFile.patch
>
>
> {{org.apache.hadoop.hbase.util.CompressionTest}} should abort execution if
> the file specified on the command line exists. This will help careless (me)
> or unsuspecting user to not loose data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12775) CompressionTest ate my HFile (sigh!)
[
https://issues.apache.org/jira/browse/HBASE-12775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aditya Kishore updated HBASE-12775:
---
Attachment: HBASE-12775-CompressionTest-ate-my-HFile.patch
> CompressionTest ate my HFile (sigh!)
>
>
> Key: HBASE-12775
> URL: https://issues.apache.org/jira/browse/HBASE-12775
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 0.98.9, 0.99.2
>Reporter: Aditya Kishore
>Assignee: Aditya Kishore
> Attachments: HBASE-12775-CompressionTest-ate-my-HFile.patch
>
>
> {{org.apache.hadoop.hbase.util.CompressionTest}} should abort execution if
> the file specified on the command line exists. This will help careless (me)
> or unsuspecting user to not loose data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12774) Fix the inconsistent permission checks for bulkloading.
[ https://issues.apache.org/jira/browse/HBASE-12774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Srungarapu updated HBASE-12774: Attachment: HBASE-12774.patch > Fix the inconsistent permission checks for bulkloading. > --- > > Key: HBASE-12774 > URL: https://issues.apache.org/jira/browse/HBASE-12774 > Project: HBase > Issue Type: Improvement >Reporter: Srikanth Srungarapu >Assignee: Srikanth Srungarapu >Priority: Minor > Attachments: HBASE-12774.patch > > > Three checks(prePrepareBulkLoad, preCleanupBulkLoad, preBulkLoadHFile) are > being done while performing secure bulk load, and it looks the former two > checks for 'W', while the later checks for 'C'. After having offline chat > with [~jdcryans], looks like the inconsistency among checks were unintended. > So, we can address this multiple ways. > * All checks should be for 'W' > * All checks should be for 'C' > * All checks should be for both 'W' and 'C' > Posting the initial patch going by the first option. Open to discussion. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12775) CompressionTest ate my HFile (sigh!)
Aditya Kishore created HBASE-12775:
--
Summary: CompressionTest ate my HFile (sigh!)
Key: HBASE-12775
URL: https://issues.apache.org/jira/browse/HBASE-12775
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.99.2, 0.98.9
Reporter: Aditya Kishore
Assignee: Aditya Kishore
{{org.apache.hadoop.hbase.util.CompressionTest}} should abort execution if the
file specified on the command line exists. This will help careless (me) or
unsuspecting user to not loose data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12723) Update ACL matrix to reflect reality
[
https://issues.apache.org/jira/browse/HBASE-12723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260538#comment-14260538
]
Srikanth Srungarapu commented on HBASE-12723:
-
Detected an inconsistency in bulkloading checks. Will post the patch, once it
gets resolved.
> Update ACL matrix to reflect reality
>
>
> Key: HBASE-12723
> URL: https://issues.apache.org/jira/browse/HBASE-12723
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Srikanth Srungarapu
> Fix For: 2.0.0, 1.0.1, 1.1.0
>
>
> The ACL matrix in the book should be updated with the recent changes.
> https://hbase.apache.org/book/appendix_acl_matrix.html
> Also the format is not optimal. There is a hierarchy relation between scopes
> (GLOBAL > NS > TABLE), but not so much between Permissions (A,C,R)
> Some things to do:
> - {{Minimum Permission}} column does not make sense. We should replace it.
> - Add information about superuser
> - grant is a multi level thing. Required permissions depend on the scope.
> - See HBASE-12511 and others changed some of the permissions
> What I would like to see at the end is something like:
> {code}
> createNamespace: superuser | global(A)
> deleteNamespace: superuser | global(A) | NS(A)
> modifyNamespace: superuser | global(A) | NS(A)
> getNamespaceDescriptor : superuser | global(A) | NS(A)
> listNamespaces : All access*
> createTable: superuser | global(C) | NS(C)
> grant
> NS Perm : superuser | global(A) | NS(A)
> Table Perm : ...
> revoke
> NS Perm : superuser | global(A) | NS(A)
> Table Perm : ...
> getPerms
> NS perm : superuser | global(A) | NS(A)
> Table Perm : ...
> {code}
> See HBASE-12511.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12774) Fix the inconsistent permission checks for bulkloading.
[ https://issues.apache.org/jira/browse/HBASE-12774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Srungarapu updated HBASE-12774: Status: Patch Available (was: Open) > Fix the inconsistent permission checks for bulkloading. > --- > > Key: HBASE-12774 > URL: https://issues.apache.org/jira/browse/HBASE-12774 > Project: HBase > Issue Type: Improvement >Reporter: Srikanth Srungarapu >Assignee: Srikanth Srungarapu >Priority: Minor > Attachments: HBASE-12774.patch > > > Three checks(prePrepareBulkLoad, preCleanupBulkLoad, preBulkLoadHFile) are > being done while performing secure bulk load, and it looks the former two > checks for 'W', while the later checks for 'C'. After having offline chat > with [~jdcryans], looks like the inconsistency among checks were unintended. > So, we can address this multiple ways. > * All checks should be for 'W' > * All checks should be for 'C' > * All checks should be for both 'W' and 'C' > Posting the initial patch going by the first option. Open to discussion. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12774) Fix the inconsistent permission checks for bulkloading.
Srikanth Srungarapu created HBASE-12774: --- Summary: Fix the inconsistent permission checks for bulkloading. Key: HBASE-12774 URL: https://issues.apache.org/jira/browse/HBASE-12774 Project: HBase Issue Type: Improvement Reporter: Srikanth Srungarapu Assignee: Srikanth Srungarapu Priority: Minor Three checks(prePrepareBulkLoad, preCleanupBulkLoad, preBulkLoadHFile) are being done while performing secure bulk load, and it looks the former two checks for 'W', while the later checks for 'C'. After having offline chat with [~jdcryans], looks like the inconsistency among checks were unintended. So, we can address this multiple ways. * All checks should be for 'W' * All checks should be for 'C' * All checks should be for both 'W' and 'C' Posting the initial patch going by the first option. Open to discussion. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12028) Abort the RegionServer, when one of it's handler threads die
[ https://issues.apache.org/jira/browse/HBASE-12028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alicia Ying Shu updated HBASE-12028: Attachment: hbase-12028-v5.patch > Abort the RegionServer, when one of it's handler threads die > > > Key: HBASE-12028 > URL: https://issues.apache.org/jira/browse/HBASE-12028 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Sudarshan Kadambi >Assignee: Alicia Ying Shu > Attachments: Hbase-12028-v3.patch, Hbase-12028.patch, > hbase-12028-v4.patch, hbase-12028-v5.patch > > > Over in HBase-11813, a user identified an issue where in all the RPC handler > threads would exit with StackOverflow errors due to an unchecked > recursion-terminating condition. Our clusters demonstrated the same trace. > While the patch posted for HBASE-11813 got our clusters to be merry again, > the breakdown surfaced some larger issues. > When the RegionServer had all it's RPC handler threads dead, it continued to > have regions assigned it. Clearly, it wouldn't be able to serve reads and > writes on those regions. A second issue was that when a user tried to disable > or drop a table, the master would try to communicate to the regionserver for > region unassignment. Since the same handler threads seem to be used for > master <-> RS communication as well, the master ended up hanging on the RS > indefinitely. Eventually, the master stopped responding to all table > meta-operations. > A handler thread should never exit, and if it does, it seems like the more > prudent thing to do would be for the RS to abort. This way, at least recovery > can be undertaken and the regions could be reassigned elsewhere. I also think > that the master<->RS communication should get its own exclusive threadpool, > but I'll wait until this issue has been sufficiently discussed before opening > an issue ticket for that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12028) Abort the RegionServer, when one of it's handler threads die
[ https://issues.apache.org/jira/browse/HBASE-12028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alicia Ying Shu updated HBASE-12028: Attachment: (was: hbase-12028-v5.patch) > Abort the RegionServer, when one of it's handler threads die > > > Key: HBASE-12028 > URL: https://issues.apache.org/jira/browse/HBASE-12028 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Sudarshan Kadambi >Assignee: Alicia Ying Shu > Attachments: Hbase-12028-v3.patch, Hbase-12028.patch, > hbase-12028-v4.patch > > > Over in HBase-11813, a user identified an issue where in all the RPC handler > threads would exit with StackOverflow errors due to an unchecked > recursion-terminating condition. Our clusters demonstrated the same trace. > While the patch posted for HBASE-11813 got our clusters to be merry again, > the breakdown surfaced some larger issues. > When the RegionServer had all it's RPC handler threads dead, it continued to > have regions assigned it. Clearly, it wouldn't be able to serve reads and > writes on those regions. A second issue was that when a user tried to disable > or drop a table, the master would try to communicate to the regionserver for > region unassignment. Since the same handler threads seem to be used for > master <-> RS communication as well, the master ended up hanging on the RS > indefinitely. Eventually, the master stopped responding to all table > meta-operations. > A handler thread should never exit, and if it does, it seems like the more > prudent thing to do would be for the RS to abort. This way, at least recovery > can be undertaken and the regions could be reassigned elsewhere. I also think > that the master<->RS communication should get its own exclusive threadpool, > but I'll wait until this issue has been sufficiently discussed before opening > an issue ticket for that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12769) Replication fails to delete all corresponding zk nodes when peer is removed
[ https://issues.apache.org/jira/browse/HBASE-12769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260527#comment-14260527 ] Enis Soztutar commented on HBASE-12769: --- i think proper solution should be doing HBASE-11392 first, then keeping the state of the "remove peer" operation in master and ensure that it is completed via HBASE-5487 or HBASE-12439. > Replication fails to delete all corresponding zk nodes when peer is removed > --- > > Key: HBASE-12769 > URL: https://issues.apache.org/jira/browse/HBASE-12769 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 0.99.2 >Reporter: cuijianwei >Priority: Minor > > When removing a peer, the client side will delete peerId under peersZNode > node; then alive region servers will be notified and delete corresponding > hlog queues under its rsZNode of replication. However, if there are failed > servers whose hlog queues have not been transferred by alive servers(this > likely happens if setting a big value to "replication.sleep.before.failover" > and lots of region servers restarted), these hlog queues won't be deleted > after the peer is removed. I think remove_peer should guarantee all > corresponding zk nodes have been removed after it completes; otherwise, if we > create a new peer with the same peerId with the removed one, there might be > unexpected data to be replicated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
