[jira] [Commented] (HBASE-4493) book.xml - moving 2 entries to newly created RegionServer section
[ https://issues.apache.org/jira/browse/HBASE-4493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115175#comment-13115175 ] Hudson commented on HBASE-4493: --- Integrated in HBase-TRUNK #2260 (See [https://builds.apache.org/job/HBase-TRUNK/2260/]) HBASE-4493 book.xml dmeil : Files : * /hbase/trunk/src/docbkx/book.xml > book.xml - moving 2 entries to newly created RegionServer section > - > > Key: HBASE-4493 > URL: https://issues.apache.org/jira/browse/HBASE-4493 > Project: HBase > Issue Type: Improvement >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Minor > Attachments: book_HBASE_4493.xml.patch > > > book.xml > * Arch section. Since RegionServer is now a top-level section under Arch, > moved existing BlockCache and WAL entries under RegionServer instead of being > under Regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4433) avoid extra next (potentially a seek) if done with column/row
[ https://issues.apache.org/jira/browse/HBASE-4433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115353#comment-13115353 ] Hudson commented on HBASE-4433: --- Integrated in HBase-TRUNK #2261 (See [https://builds.apache.org/job/HBase-TRUNK/2261/]) HBASE-4433 avoid extra next (potentially a seek) if done with column/row (kannan via jgray) jgray : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestExplicitColumnTracker.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestQueryMatcher.java > avoid extra next (potentially a seek) if done with column/row > - > > Key: HBASE-4433 > URL: https://issues.apache.org/jira/browse/HBASE-4433 > Project: HBase > Issue Type: Improvement >Reporter: Kannan Muthukkaruppan >Assignee: Kannan Muthukkaruppan > Fix For: 0.94.0 > > > [Noticed this in 89, but quite likely true of trunk as well.] > When we are done with the requested column(s) the code still does an extra > next() call before it realizes that it is actually done. This extra next() > call could potentially result in an unnecessary extra block load. This is > likely to be especially bad for CFs where the KVs are large blobs where each > KV may be occupying a block of its own. So the next() can often load a new > unrelated block unnecessarily. > -- > For the simple case of reading say the top-most column in a row in a single > file, where each column (KV) was say a block of its own-- it seems that we > are reading 3 blocks, instead of 1 block! > I am working on a simple patch and with that the number of seeks is down to > 2. > [There is still an extra seek left. I think there were two levels of > extra/unnecessary next() we were doing without actually confirming that the > next was needed. One at the StoreScanner/ScanQueryMatcher level which this > diff avoids. I think the other is at hfs.next() (at the storefile scanner > level) that's happening whenever a HFile scanner servers out a data-- and > perhaps that's the additional seek that we need to avoid. But I want to > tackle this optimization first as the two issues seem unrelated.] > -- > The basic idea of the patch I am working on/testing is as follows. The > ExplicitColumnTracker currently returns "INCLUDE" to the ScanQueryMatcher if > the KV needs to be included and then if done, only in the the next call it > returns the appropriate SEEK_NEXT_COL or SEEK_NEXT_ROW hint. For the cases > when ExplicitColumnTracker knows it is done with a particular column/row, the > patch attempts to combine the INCLUDE code and done hint into a single match > code-- INCLUDE_AND_SEEK_NEXT_COL and INCLUDE_AND_SEEK_NEXT_ROW. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4433) avoid extra next (potentially a seek) if done with column/row
[ https://issues.apache.org/jira/browse/HBASE-4433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115938#comment-13115938 ] Hudson commented on HBASE-4433: --- Integrated in HBase-0.92 #23 (See [https://builds.apache.org/job/HBase-0.92/23/]) HBASE-4433: avoid extra next (potentially a seek) if done with column/row HBASE-4433: avoid extra next (potentially a seek) if done with column/row stack : Files : * /hbase/branches/0.92/CHANGES.txt stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestExplicitColumnTracker.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestQueryMatcher.java > avoid extra next (potentially a seek) if done with column/row > - > > Key: HBASE-4433 > URL: https://issues.apache.org/jira/browse/HBASE-4433 > Project: HBase > Issue Type: Improvement >Reporter: Kannan Muthukkaruppan >Assignee: Kannan Muthukkaruppan > Fix For: 0.92.0 > > > [Noticed this in 89, but quite likely true of trunk as well.] > When we are done with the requested column(s) the code still does an extra > next() call before it realizes that it is actually done. This extra next() > call could potentially result in an unnecessary extra block load. This is > likely to be especially bad for CFs where the KVs are large blobs where each > KV may be occupying a block of its own. So the next() can often load a new > unrelated block unnecessarily. > -- > For the simple case of reading say the top-most column in a row in a single > file, where each column (KV) was say a block of its own-- it seems that we > are reading 3 blocks, instead of 1 block! > I am working on a simple patch and with that the number of seeks is down to > 2. > [There is still an extra seek left. I think there were two levels of > extra/unnecessary next() we were doing without actually confirming that the > next was needed. One at the StoreScanner/ScanQueryMatcher level which this > diff avoids. I think the other is at hfs.next() (at the storefile scanner > level) that's happening whenever a HFile scanner servers out a data-- and > perhaps that's the additional seek that we need to avoid. But I want to > tackle this optimization first as the two issues seem unrelated.] > -- > The basic idea of the patch I am working on/testing is as follows. The > ExplicitColumnTracker currently returns "INCLUDE" to the ScanQueryMatcher if > the KV needs to be included and then if done, only in the the next call it > returns the appropriate SEEK_NEXT_COL or SEEK_NEXT_ROW hint. For the cases > when ExplicitColumnTracker knows it is done with a particular column/row, the > patch attempts to combine the INCLUDE code and done hint into a single match > code-- INCLUDE_AND_SEEK_NEXT_COL and INCLUDE_AND_SEEK_NEXT_ROW. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4455) Rolling restart RSs scenario, -ROOT-, .META. regions are lost in AssignmentManager
[ https://issues.apache.org/jira/browse/HBASE-4455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116001#comment-13116001 ] Hudson commented on HBASE-4455: --- Integrated in HBase-TRUNK #2262 (See [https://builds.apache.org/job/HBase-TRUNK/2262/]) HBASE-4455 Rolling restart RSs scenario, -ROOT-, .META. regions are lost in AssignmentManager (Ming Ma) tedyu : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/MasterAddressTracker.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/handler/CloseRegionHandler.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ClusterStatusTracker.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperNodeTracker.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperNodeTracker.java > Rolling restart RSs scenario, -ROOT-, .META. regions are lost in > AssignmentManager > -- > > Key: HBASE-4455 > URL: https://issues.apache.org/jira/browse/HBASE-4455 > Project: HBase > Issue Type: Bug >Reporter: Ming Ma >Assignee: Ming Ma > Fix For: 0.92.0 > > > Keep Master up all the time, do rolling restart of RSs like this - stop RS1, > wait for 2 seconds, stop RS2, start RS1, wait for 2 seconds, stop RS3, start > RS2, wait for 2 seconds, etc. After a while, you will find the -ROOT-, .META. > regions aren't in "regions in transtion" from AssignmentManager point of > view, but they aren't assigned to any regions. Here are the issues. > 1. .-ROOT- or .META. location is stale when MetaServerShutdownHandler is > invoked to check if it contains -ROOT- region. That is due to long delay from > ZK notification and async nature of the system. Here is an example, even > though new root region server sea-lab-1,60020,1316380133656 is set at T2, at > T3 the shutdown process for sea-lab-1,60020,1316380133656, the root location > still points to old server sea-lab-3,60020,1316380037898. > T1: 2011-09-18 14:08:52,470 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: > master:6 > -0x1327e43175e Retrieved 29 byte(s) of data from znode > /hbase/root-regio > n-server and set watcher; sea-lab-3,60020,1316380037898 > T2: 2011-09-18 14:08:57,173 INFO > org.apache.hadoop.hbase.catalog.RootLocationEditor: Setting ROOT region > location in ZooKeeper as sea-lab-1,60020,1316380133656 > T3: 2011-09-18 14:10:26,393 DEBUG > org.apache.hadoop.hbase.master.ServerManager: Adde > d=sea-lab-1,60020,1316380133656 to dead servers, submitted shutdown handler > to be executed, root=false, meta=true, current Root Location: > sea-lab-3,60020,1316380037898 > T4: 2011-09-18 14:12:37,314 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: > master:6 > -0x1327e43175e Retrieved 29 byte(s) of data from znode > /hbase/root-region-server and set watcher; sea-lab-1,60020,1316380133656 > 2. The MetaServerShutdownHandler worker thread that waits for -ROOT- or > .META. availability could be blocked. If meanwhile, the new server that > -ROOT- or .META. is being assigned restarted, another instance of > MetaServerShutdownHandler is queued. Eventually, all > MetaServerShutdownHandler worker threads are filled up. It looks like > HBASE-4245. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4455) Rolling restart RSs scenario, -ROOT-, .META. regions are lost in AssignmentManager
[ https://issues.apache.org/jira/browse/HBASE-4455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116059#comment-13116059 ] Hudson commented on HBASE-4455: --- Integrated in HBase-0.92 #24 (See [https://builds.apache.org/job/HBase-0.92/24/]) HBASE-4455 Rolling restart RSs scenario, -ROOT-, .META. regions are lost in AssignmentManager (Ming Ma) tedyu : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/MasterAddressTracker.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/handler/CloseRegionHandler.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/ClusterStatusTracker.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperNodeTracker.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperNodeTracker.java > Rolling restart RSs scenario, -ROOT-, .META. regions are lost in > AssignmentManager > -- > > Key: HBASE-4455 > URL: https://issues.apache.org/jira/browse/HBASE-4455 > Project: HBase > Issue Type: Bug >Reporter: Ming Ma >Assignee: Ming Ma > Fix For: 0.92.0 > > > Keep Master up all the time, do rolling restart of RSs like this - stop RS1, > wait for 2 seconds, stop RS2, start RS1, wait for 2 seconds, stop RS3, start > RS2, wait for 2 seconds, etc. After a while, you will find the -ROOT-, .META. > regions aren't in "regions in transtion" from AssignmentManager point of > view, but they aren't assigned to any regions. Here are the issues. > 1. .-ROOT- or .META. location is stale when MetaServerShutdownHandler is > invoked to check if it contains -ROOT- region. That is due to long delay from > ZK notification and async nature of the system. Here is an example, even > though new root region server sea-lab-1,60020,1316380133656 is set at T2, at > T3 the shutdown process for sea-lab-1,60020,1316380133656, the root location > still points to old server sea-lab-3,60020,1316380037898. > T1: 2011-09-18 14:08:52,470 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: > master:6 > -0x1327e43175e Retrieved 29 byte(s) of data from znode > /hbase/root-regio > n-server and set watcher; sea-lab-3,60020,1316380037898 > T2: 2011-09-18 14:08:57,173 INFO > org.apache.hadoop.hbase.catalog.RootLocationEditor: Setting ROOT region > location in ZooKeeper as sea-lab-1,60020,1316380133656 > T3: 2011-09-18 14:10:26,393 DEBUG > org.apache.hadoop.hbase.master.ServerManager: Adde > d=sea-lab-1,60020,1316380133656 to dead servers, submitted shutdown handler > to be executed, root=false, meta=true, current Root Location: > sea-lab-3,60020,1316380037898 > T4: 2011-09-18 14:12:37,314 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil: > master:6 > -0x1327e43175e Retrieved 29 byte(s) of data from znode > /hbase/root-region-server and set watcher; sea-lab-1,60020,1316380133656 > 2. The MetaServerShutdownHandler worker thread that waits for -ROOT- or > .META. availability could be blocked. If meanwhile, the new server that > -ROOT- or .META. is being assigned restarted, another instance of > MetaServerShutdownHandler is queued. Eventually, all > MetaServerShutdownHandler worker threads are filled up. It looks like > HBASE-4245. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4488) Store could miss rows during flush
[ https://issues.apache.org/jira/browse/HBASE-4488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116663#comment-13116663 ] Hudson commented on HBASE-4488: --- Integrated in HBase-0.92 #25 (See [https://builds.apache.org/job/HBase-0.92/25/]) HBASE-4488 Store could miss rows during flush (Lars H via jgray) jgray : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java > Store could miss rows during flush > -- > > Key: HBASE-4488 > URL: https://issues.apache.org/jira/browse/HBASE-4488 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Affects Versions: 0.92.0, 0.94.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.92.0 > > Attachments: 4488.txt > > > While looking at HBASE-4344 I found that my change HBASE-4241 contains a > critical mistake: > The while(scanner.next(kvs)) loop is incorrect and might miss the last edits. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4131) Make the Replication Service pluggable via a standard interface definition
[ https://issues.apache.org/jira/browse/HBASE-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116762#comment-13116762 ] Hudson commented on HBASE-4131: --- Integrated in HBase-TRUNK #2264 (See [https://builds.apache.org/job/HBase-TRUNK/2264/]) HBASE-4131 Make the Replication Service pluggable via a standard interface definition (dhruba via jgray) jgray : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ReplicationService.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ReplicationSinkService.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ReplicationSourceService.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java > Make the Replication Service pluggable via a standard interface definition > -- > > Key: HBASE-4131 > URL: https://issues.apache.org/jira/browse/HBASE-4131 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: dhruba borthakur >Assignee: dhruba borthakur > Fix For: 0.94.0 > > Attachments: 4131-backedout.txt, replicationInterface1.txt, > replicationInterface2.txt, replicationInterface3.txt, > replicationInterface4.txt > > > The current HBase code supports a replication service that can be used to > sync data from from one hbase cluster to another. It would be nice to make it > a pluggable interface so that other cross-data-center replication services > can be used in conjuction with HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4504) book.xml - adding section on filters
[ https://issues.apache.org/jira/browse/HBASE-4504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116763#comment-13116763 ] Hudson commented on HBASE-4504: --- Integrated in HBase-TRUNK #2264 (See [https://builds.apache.org/job/HBase-TRUNK/2264/]) HBASE-4504 book.xml - filters dmeil : Files : * /hbase/trunk/src/docbkx/book.xml > book.xml - adding section on filters > > > Key: HBASE-4504 > URL: https://issues.apache.org/jira/browse/HBASE-4504 > Project: HBase > Issue Type: Improvement >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Minor > Attachments: book_HBASE_4504.xml.patch > > > Under Architecture > * new sub-section for Client Filters, with sub-sections by filter-type, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4488) Store could miss rows during flush
[ https://issues.apache.org/jira/browse/HBASE-4488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116761#comment-13116761 ] Hudson commented on HBASE-4488: --- Integrated in HBase-TRUNK #2264 (See [https://builds.apache.org/job/HBase-TRUNK/2264/]) HBASE-4488 Store could miss rows during flush (Lars H via jgray) jgray : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java > Store could miss rows during flush > -- > > Key: HBASE-4488 > URL: https://issues.apache.org/jira/browse/HBASE-4488 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Affects Versions: 0.92.0, 0.94.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.92.0 > > Attachments: 4488.txt > > > While looking at HBASE-4344 I found that my change HBASE-4241 contains a > critical mistake: > The while(scanner.next(kvs)) loop is incorrect and might miss the last edits. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4454) Add failsafe plugin to build and rename integration tests
[ https://issues.apache.org/jira/browse/HBASE-4454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13117115#comment-13117115 ] Hudson commented on HBASE-4454: --- Integrated in HBase-TRUNK #2267 (See [https://builds.apache.org/job/HBase-TRUNK/2267/]) HBASE-4454 Add failsafe plugin to build and rename integration tests stack : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/pom.xml * /hbase/trunk/src/docbkx/developer.xml > Add failsafe plugin to build and rename integration tests > - > > Key: HBASE-4454 > URL: https://issues.apache.org/jira/browse/HBASE-4454 > Project: HBase > Issue Type: Improvement >Reporter: Jesse Yates > Fix For: 0.92.0 > > Attachments: mvn_HBASE-4454.patch > > > Add the maven-failsafe-plugin to the build process so we can run integration > tests with "mvn verify". This will also involve a renaming of integration > tests to conform to a new integration test regex. > This is a stopgap measure while we until break them out into their own module. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4454) Add failsafe plugin to build and rename integration tests
[ https://issues.apache.org/jira/browse/HBASE-4454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13117194#comment-13117194 ] Hudson commented on HBASE-4454: --- Integrated in HBase-0.92 #27 (See [https://builds.apache.org/job/HBase-0.92/27/]) HBASE-4454 Add failsafe plugin to build and rename integration tests stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/pom.xml > Add failsafe plugin to build and rename integration tests > - > > Key: HBASE-4454 > URL: https://issues.apache.org/jira/browse/HBASE-4454 > Project: HBase > Issue Type: Improvement >Reporter: Jesse Yates > Fix For: 0.92.0 > > Attachments: mvn_HBASE-4454.patch > > > Add the maven-failsafe-plugin to the build process so we can run integration > tests with "mvn verify". This will also involve a renaming of integration > tests to conform to a new integration test regex. > This is a stopgap measure while we until break them out into their own module. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4492) TestRollingRestart fails intermittently
[
https://issues.apache.org/jira/browse/HBASE-4492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13117230#comment-13117230
]
Hudson commented on HBASE-4492:
---
Integrated in HBase-TRUNK #2268 (See
[https://builds.apache.org/job/HBase-TRUNK/2268/])
HBASE-4492 TestRollingRestart Fails Intermittently(Ted yu and Ram)
ramkrishna :
Files :
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestRollingRestart.java
> TestRollingRestart fails intermittently
> ---
>
> Key: HBASE-4492
> URL: https://issues.apache.org/jira/browse/HBASE-4492
> Project: HBase
> Issue Type: Test
>Affects Versions: 0.92.0
>Reporter: Ted Yu
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.0
>
> Attachments: 4492-v2.txt, 4492.txt, HBASE-4492.patch
>
>
> I got the following when running test suite on TRUNK:
> {code}
> testBasicRollingRestart(org.apache.hadoop.hbase.master.TestRollingRestart)
> Time elapsed: 300.28 sec <<< ERROR!
> java.lang.Exception: test timed out after 30 milliseconds
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.hbase.master.TestRollingRestart.waitForRSShutdownToStartAndFinish(TestRollingRestart.java:313)
> at
> org.apache.hadoop.hbase.master.TestRollingRestart.testBasicRollingRestart(TestRollingRestart.java:210)
> {code}
> I ran TestRollingRestart#testBasicRollingRestart manually afterwards which
> wiped out test output file for the failed test.
> Similar failure can be found on Jenkins:
> https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.92/19/testReport/junit/org.apache.hadoop.hbase.master/TestRollingRestart/testBasicRollingRestart/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4492) TestRollingRestart fails intermittently
[
https://issues.apache.org/jira/browse/HBASE-4492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13117344#comment-13117344
]
Hudson commented on HBASE-4492:
---
Integrated in HBase-0.92 #28 (See
[https://builds.apache.org/job/HBase-0.92/28/])
HBASE-4492 TestMasterFailover fails intermittently(Ted yu and Ram)
ramkrishna :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
*
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestRollingRestart.java
> TestRollingRestart fails intermittently
> ---
>
> Key: HBASE-4492
> URL: https://issues.apache.org/jira/browse/HBASE-4492
> Project: HBase
> Issue Type: Test
>Affects Versions: 0.92.0
>Reporter: Ted Yu
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.0
>
> Attachments: 4492-v2.txt, 4492.txt, HBASE-4492.patch
>
>
> I got the following when running test suite on TRUNK:
> {code}
> testBasicRollingRestart(org.apache.hadoop.hbase.master.TestRollingRestart)
> Time elapsed: 300.28 sec <<< ERROR!
> java.lang.Exception: test timed out after 30 milliseconds
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.hbase.master.TestRollingRestart.waitForRSShutdownToStartAndFinish(TestRollingRestart.java:313)
> at
> org.apache.hadoop.hbase.master.TestRollingRestart.testBasicRollingRestart(TestRollingRestart.java:210)
> {code}
> I ran TestRollingRestart#testBasicRollingRestart manually afterwards which
> wiped out test output file for the failed test.
> Similar failure can be found on Jenkins:
> https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.92/19/testReport/junit/org.apache.hadoop.hbase.master/TestRollingRestart/testBasicRollingRestart/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4513) NOTICES.txt refers to Facebook for Thrift
[
https://issues.apache.org/jira/browse/HBASE-4513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13117604#comment-13117604
]
Hudson commented on HBASE-4513:
---
Integrated in HBase-TRUNK #2269 (See
[https://builds.apache.org/job/HBase-TRUNK/2269/])
HBASE-4513 NOTICES.txt refers to Facebook for Thrift
stack :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/NOTICE.txt
> NOTICES.txt refers to Facebook for Thrift
> -
>
> Key: HBASE-4513
> URL: https://issues.apache.org/jira/browse/HBASE-4513
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Joe Pallas
>Assignee: stack
>Priority: Minor
> Fix For: 0.92.0
>
>
> The NOTICES.txt file says:
> {quote}
> In addition, this product includes software developed by:
> Facebook, Inc. (http://developers.facebook.com/thrift/ -- Page includes the
> Thrift Software License)
> {quote}
> Since Thrift is now an Apache project, this seems to be obsolete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4512) JVMClusterUtil throwing wrong exception when master thread cannot be created
[
https://issues.apache.org/jira/browse/HBASE-4512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13117603#comment-13117603
]
Hudson commented on HBASE-4512:
---
Integrated in HBase-TRUNK #2269 (See
[https://builds.apache.org/job/HBase-TRUNK/2269/])
HBASE-4512 JVMClusterUtil throwing wrong exception when master thread
cannot be created (Ram) - Did not update the CHANGES.txt
HBASE-4512 JVMClusterUtil throwing wrong exception when master thread cannot be
created (Ram)
ramkrishna :
Files :
* /hbase/trunk/CHANGES.txt
ramkrishna :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java
> JVMClusterUtil throwing wrong exception when master thread cannot be created
>
>
> Key: HBASE-4512
> URL: https://issues.apache.org/jira/browse/HBASE-4512
> Project: HBase
> Issue Type: Bug
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Trivial
> Fix For: 0.92.0, 0.94.0, 0.90.5
>
> Attachments: HBASE-4512_trunk.patch, HBASE_4253_0.90.patch
>
>
> In JVMClusterUtil.MasterThread createMasterThread
> {code}
> throw new RuntimeException("Failed construction of RegionServer: " +
> {code}
> It must be failed construction of Master.
> Very trivial one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4512) JVMClusterUtil throwing wrong exception when master thread cannot be created
[
https://issues.apache.org/jira/browse/HBASE-4512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13117706#comment-13117706
]
Hudson commented on HBASE-4512:
---
Integrated in HBase-0.92 #30 (See
[https://builds.apache.org/job/HBase-0.92/30/])
HBASE-4512 JVMClusterUtil throwing wrong exception when master thread
cannot be created (Ram)
ramkrishna :
Files :
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java
> JVMClusterUtil throwing wrong exception when master thread cannot be created
>
>
> Key: HBASE-4512
> URL: https://issues.apache.org/jira/browse/HBASE-4512
> Project: HBase
> Issue Type: Bug
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Trivial
> Fix For: 0.92.0, 0.94.0, 0.90.5
>
> Attachments: HBASE-4512_trunk.patch, HBASE_4253_0.90.patch
>
>
> In JVMClusterUtil.MasterThread createMasterThread
> {code}
> throw new RuntimeException("Failed construction of RegionServer: " +
> {code}
> It must be failed construction of Master.
> Very trivial one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4513) NOTICES.txt refers to Facebook for Thrift
[
https://issues.apache.org/jira/browse/HBASE-4513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13117707#comment-13117707
]
Hudson commented on HBASE-4513:
---
Integrated in HBase-0.92 #30 (See
[https://builds.apache.org/job/HBase-0.92/30/])
HBASE-4513 NOTICES.txt refers to Facebook for Thrift
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/NOTICE.txt
> NOTICES.txt refers to Facebook for Thrift
> -
>
> Key: HBASE-4513
> URL: https://issues.apache.org/jira/browse/HBASE-4513
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Joe Pallas
>Assignee: stack
>Priority: Minor
> Fix For: 0.92.0
>
>
> The NOTICES.txt file says:
> {quote}
> In addition, this product includes software developed by:
> Facebook, Inc. (http://developers.facebook.com/thrift/ -- Page includes the
> Thrift Software License)
> {quote}
> Since Thrift is now an Apache project, this seems to be obsolete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13117791#comment-13117791 ] Hudson commented on HBASE-4477: --- Integrated in HBase-0.92 #31 (See [https://builds.apache.org/job/HBase-0.92/31/]) HBASE-4477 Ability for an application to store metadata into the transaction log (dhruba via jgray) jgray : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/Mutation.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverBypass.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverStacking.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerCoprocessorExceptionWithAbort.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerCoprocessorExceptionWithRemove.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/replication/TestMasterReplication.java > Ability for an application to store metadata into the transaction log > - > > Key: HBASE-4477 > URL: https://issues.apache.org/jira/browse/HBASE-4477 > Project: HBase > Issue Type: Improvement >Reporter: dhruba borthakur >Assignee: dhruba borthakur > Fix For: 0.92.0 > > Attachments: coprocessorPut1.txt, coprocessorPut2.txt, > hlogMetadata1.txt > > > mySQL allows an application to store an arbitrary blob along with each > transaction in its transaction logs. This JIRA is to have a similar feature > request for HBASE. > The use case is as follows: An application on one data center A stores a blob > of data along with each transaction. A replication software picks up these > blobs from the transaction logs in A and hands it to another instance of the > same application running on a remote data center B. The application in B is > responsible for applying this to the remote Hbase cluster (and also handle > conflict resolution if any). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4477) Ability for an application to store metadata into the transaction log
[ https://issues.apache.org/jira/browse/HBASE-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13117840#comment-13117840 ] Hudson commented on HBASE-4477: --- Integrated in HBase-TRUNK #2270 (See [https://builds.apache.org/job/HBase-TRUNK/2270/]) HBASE-4477 Ability for an application to store metadata into the transaction log (dhruba via jgray) jgray : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Mutation.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverBypass.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverStacking.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerCoprocessorExceptionWithAbort.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerCoprocessorExceptionWithRemove.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/replication/TestMasterReplication.java > Ability for an application to store metadata into the transaction log > - > > Key: HBASE-4477 > URL: https://issues.apache.org/jira/browse/HBASE-4477 > Project: HBase > Issue Type: Improvement >Reporter: dhruba borthakur >Assignee: dhruba borthakur > Fix For: 0.92.0 > > Attachments: coprocessorPut1.txt, coprocessorPut2.txt, > hlogMetadata1.txt > > > mySQL allows an application to store an arbitrary blob along with each > transaction in its transaction logs. This JIRA is to have a similar feature > request for HBASE. > The use case is as follows: An application on one data center A stores a blob > of data along with each transaction. A replication software picks up these > blobs from the transaction logs in A and hands it to another instance of the > same application running on a remote data center B. The application in B is > responsible for applying this to the remote Hbase cluster (and also handle > conflict resolution if any). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4212) TestMasterFailover fails occasionally
[
https://issues.apache.org/jira/browse/HBASE-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118041#comment-13118041
]
Hudson commented on HBASE-4212:
---
Integrated in HBase-0.92 #33 (See
[https://builds.apache.org/job/HBase-0.92/33/])
HBASE-4212 TestMasterFailover fails occasionally (Gao Jinchao)
tedyu :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
> TestMasterFailover fails occasionally
> -
>
> Key: HBASE-4212
> URL: https://issues.apache.org/jira/browse/HBASE-4212
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.90.4
>Reporter: gaojinchao
>Assignee: gaojinchao
> Fix For: 0.90.5
>
> Attachments: HBASE-4212_TrunkV1.patch, HBASE-4212_branch90V1.patch
>
>
> It seems a bug. The root in RIT can't be moved..
> In the failover process, it enforces root on-line. But not clean zk node.
> test will wait forever.
> void processFailover() throws KeeperException, IOException,
> InterruptedException {
>
> // we enforce on-line root.
> HServerInfo hsi =
>
> this.serverManager.getHServerInfo(this.catalogTracker.getMetaLocation());
> regionOnline(HRegionInfo.FIRST_META_REGIONINFO, hsi);
> hsi =
> this.serverManager.getHServerInfo(this.catalogTracker.getRootLocation());
> regionOnline(HRegionInfo.ROOT_REGIONINFO, hsi);
> It seems that we should wait finished as meta region
> int assignRootAndMeta()
> throws InterruptedException, IOException, KeeperException {
> int assigned = 0;
> long timeout = this.conf.getLong("hbase.catalog.verification.timeout",
> 1000);
> // Work on ROOT region. Is it in zk in transition?
> boolean rit = this.assignmentManager.
>
> processRegionInTransitionAndBlockUntilAssigned(HRegionInfo.ROOT_REGIONINFO);
> if (!catalogTracker.verifyRootRegionLocation(timeout)) {
> this.assignmentManager.assignRoot();
> this.catalogTracker.waitForRoot();
> //we need add this code and guarantee that the transition has completed
> this.assignmentManager.waitForAssignment(HRegionInfo.ROOT_REGIONINFO);
> assigned++;
> }
> logs:
> 2011-08-16 07:45:40,715 DEBUG
> [RegionServer:0;C4S2.site,47710,1313495126115-EventThread]
> zookeeper.ZooKeeperWatcher(252): regionserver:47710-0x131d2690f780004
> Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected,
> path=/hbase/unassigned/70236052
> 2011-08-16 07:45:40,715 DEBUG [RS_OPEN_ROOT-C4S2.site,47710,1313495126115-0]
> zookeeper.ZKAssign(712): regionserver:47710-0x131d2690f780004 Successfully
> transitioned node 70236052 from RS_ZK_REGION_OPENING to RS_ZK_REGION_OPENING
> 2011-08-16 07:45:40,715 DEBUG [Thread-760-EventThread]
> zookeeper.ZooKeeperWatcher(252): master:60701-0x131d2690f780009 Received
> ZooKeeper Event, type=NodeDataChanged, state=SyncConnected,
> path=/hbase/unassigned/70236052
> 2011-08-16 07:45:40,716 INFO [PostOpenDeployTasks:70236052]
> catalog.RootLocationEditor(62): Setting ROOT region location in ZooKeeper as
> C4S2.site:47710
> 2011-08-16 07:45:40,716 DEBUG [Thread-760-EventThread]
> zookeeper.ZKUtil(1109): master:60701-0x131d2690f780009 Retrieved 52 byte(s)
> of data from znode /hbase/unassigned/70236052 and set watcher;
> region=-ROOT-,,0, server=C4S2.site,47710,1313495126115,
> state=RS_ZK_REGION_OPENING
> 2011-08-16 07:45:40,717 DEBUG [Thread-760-EventThread]
> master.AssignmentManager(477): Handling transition=RS_ZK_REGION_OPENING,
> server=C4S2.site,47710,1313495126115, region=70236052/-ROOT-
> 2011-08-16 07:45:40,725 DEBUG [RS_OPEN_ROOT-C4S2.site,47710,1313495126115-0]
> zookeeper.ZKAssign(661): regionserver:47710-0x131d2690f780004 Attempting to
> transition node 70236052/-ROOT- from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENED
> 2011-08-16 07:45:40,727 DEBUG [RS_OPEN_ROOT-C4S2.site,47710,1313495126115-0]
> zookeeper.ZKUtil(1109): regionserver:47710-0x131d2690f780004 Retrieved 52
> byte(s) of data from znode /hbase/unassigned/70236052; data=region=-ROOT-,,0,
> server=C4S2.site,47710,1313495126115, state=RS_ZK_REGION_OPENING
> 2011-08-16 07:45:40,740 DEBUG
> [RegionServer:0;C4S2.site,47710,1313495126115-EventThread]
> zookeeper.ZooKeeperWatcher(252): regionserver:47710-0x131d2690f780004
> Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected,
> path=/hbase/unassigned/70236052
> 2011-08-16 07:45:40,740 DEBUG [Thread-760-EventThread]
> zookeeper.ZooKeeperWatcher(252): master:60701-0x131d2690f780009 Received
> ZooKeeper Event, type=NodeDataChanged, state=SyncConnected,
> path=/hbase/unassigned/70236052
> 2011-08-16 07:45:40,740 DEBUG [RS_OPEN_ROOT-C4S2.site,47710,1313495126115-0]
> zookeeper.ZKAssign(712): regionserver:47710-0x13
[jira] [Commented] (HBASE-3130) [replication] ReplicationSource can't recover from session expired on remote clusters
[ https://issues.apache.org/jira/browse/HBASE-3130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118042#comment-13118042 ] Hudson commented on HBASE-3130: --- Integrated in HBase-0.92 #33 (See [https://builds.apache.org/job/HBase-0.92/33/]) HBASE-3130 [replication] ReplicationSource can't recover from session expired on remote clusters (Chris Trezzo via JD) HBASE-4499 [replication] Source shouldn't update ZK if it didn't progress (Chris Trezzo via JD) jdcryans : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationPeer.java > [replication] ReplicationSource can't recover from session expired on remote > clusters > - > > Key: HBASE-3130 > URL: https://issues.apache.org/jira/browse/HBASE-3130 > Project: HBase > Issue Type: Bug > Components: replication >Affects Versions: 0.92.0 >Reporter: Jean-Daniel Cryans >Assignee: Chris Trezzo > Fix For: 0.92.0 > > Attachments: 3130-v2.txt, 3130-v3.txt, 3130-v4.txt, 3130.txt, > HBASE-3130-v5.patch > > > Currently ReplicationSource cannot recover when its zookeeper connection to > its remote cluster expires. HLogs are still being tracked, but a cluster > restart is required to continue replication (or a rolling restart). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4412) No need to retry scan operation on the same server in case of RegionServerStoppedException
[
https://issues.apache.org/jira/browse/HBASE-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118040#comment-13118040
]
Hudson commented on HBASE-4412:
---
Integrated in HBase-0.92 #33 (See
[https://builds.apache.org/job/HBase-0.92/33/])
HBASE-4412 No need to retry scan operation on the same server in case of
RegionServerStoppedException (Ming Ma)
tedyu :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/HTable.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java
> No need to retry scan operation on the same server in case of
> RegionServerStoppedException
> --
>
> Key: HBASE-4412
> URL: https://issues.apache.org/jira/browse/HBASE-4412
> Project: HBase
> Issue Type: Bug
>Reporter: Ming Ma
>Assignee: Ming Ma
> Fix For: 0.92.0
>
> Attachments: HBASE-4412-trunk.patch
>
>
> ScannerCallable.java doesn't retry on the same server in case of
> NotServingRegionException.
> if (ioe instanceof NotServingRegionException) {
> // Throw a DNRE so that we break out of cycle of calling NSRE
> // when what we need is to open scanner against new location.
> // Attach NSRE to signal client that it needs to resetup scanner.
> throw new DoNotRetryIOException("Reset scanner", ioe);
> In the scenario when region server is in shutdown process,
> RegionServerStoppedException will be thrown. ScannerCallable still retry. It
> isn't necessary.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4499) [replication] Source shouldn't update ZK if it didn't progress
[ https://issues.apache.org/jira/browse/HBASE-4499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118039#comment-13118039 ] Hudson commented on HBASE-4499: --- Integrated in HBase-0.92 #33 (See [https://builds.apache.org/job/HBase-0.92/33/]) HBASE-3130 [replication] ReplicationSource can't recover from session expired on remote clusters (Chris Trezzo via JD) HBASE-4499 [replication] Source shouldn't update ZK if it didn't progress (Chris Trezzo via JD) jdcryans : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationPeer.java > [replication] Source shouldn't update ZK if it didn't progress > -- > > Key: HBASE-4499 > URL: https://issues.apache.org/jira/browse/HBASE-4499 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.90.4 >Reporter: Jean-Daniel Cryans >Assignee: Chris Trezzo >Priority: Minor > Fix For: 0.92.0 > > Attachments: 4499.txt > > > A relatively minor optimization to be done in ReplicationSource, currently it > calls ReplicationSourceManager.logPositionAndCleanOldLogs whether it made > progress or not, generating more load on ZK than necessary. The last position > should be kept around so that we can compare. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4145) Provide metrics for hbase client
[ https://issues.apache.org/jira/browse/HBASE-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118134#comment-13118134 ] Hudson commented on HBASE-4145: --- Integrated in HBase-TRUNK #2272 (See [https://builds.apache.org/job/HBase-TRUNK/2272/]) HBASE-4145 Provide metrics for hbase client, add ScanMetrics.java HBASE-4145 Provide metrics for hbase client (Ming Ma) tedyu : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/metrics * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/metrics/ScanMetrics.java tedyu : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Scan.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/TableInputFormatBase.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReader.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapred/TestTableInputFormat.java > Provide metrics for hbase client > > > Key: HBASE-4145 > URL: https://issues.apache.org/jira/browse/HBASE-4145 > Project: HBase > Issue Type: Improvement >Reporter: Ming Ma >Assignee: Ming Ma > Fix For: 0.94.0 > > Attachments: HBaseClientSideMetrics.jpg > > > Sometimes it is useful to get some metrics from hbase client point of view. > This will help understand the metrics for scan/TableInputFormat map job > scenario. > What to capture, for example, for each ResultScanner object, > 1. The number of RPC calls to RSs. > 2. The delta time between consecutive RPC calls in the current serialized > scan implementation. > 3. The number of RPC retry to RSs. > 4. The number of NotServingRegionException got. > 5. The number of remote RPC calls. This excludes those call that hbase client > calls the RS on the same machine. > 6. The number of regions accessed. > How to capture > 1. Metrics framework works for a fixed number of metrics. It doesn't fit this > scenario. > 2. Use some TBD solution in HBase to capture such dynamic metrics. If we > assume there is a solution in HBase that HBase client can use to log such > kind of metrics, TableInputFormat can pass in mapreduce task ID as > application scan ID to HBase client as small addition to existing scan API; > and HBase client can log metrics accordingly with such ID. That will allow > query, analysis later on the metrics data for specific map reduce job. > 3. Expose via MapReduce counter. It lacks certain features, for example, > there is no good way to access the metrics on per map instance; the MapReduce > framework only performs sum on the counter values so it is tricky to find the > max of certain metrics in all mapper instances. However, it might be good > enough for now. With this approach, the metrics value will be available via > MapReduce counter. > a) Have ResultScanner return a new ResultScannerMetrics interface. > b) TableInputFormat will access data from ResultScannerMetrics and populate > MapReduce counters accordingly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4499) [replication] Source shouldn't update ZK if it didn't progress
[ https://issues.apache.org/jira/browse/HBASE-4499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118132#comment-13118132 ] Hudson commented on HBASE-4499: --- Integrated in HBase-TRUNK #2272 (See [https://builds.apache.org/job/HBase-TRUNK/2272/]) HBASE-3130 [replication] ReplicationSource can't recover from session expired on remote clusters (Chris Trezzo via JD) HBASE-4499 [replication] Source shouldn't update ZK if it didn't progress (Chris Trezzo via JD) jdcryans : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationPeer.java > [replication] Source shouldn't update ZK if it didn't progress > -- > > Key: HBASE-4499 > URL: https://issues.apache.org/jira/browse/HBASE-4499 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.90.4 >Reporter: Jean-Daniel Cryans >Assignee: Chris Trezzo >Priority: Minor > Fix For: 0.92.0 > > Attachments: 4499.txt > > > A relatively minor optimization to be done in ReplicationSource, currently it > calls ReplicationSourceManager.logPositionAndCleanOldLogs whether it made > progress or not, generating more load on ZK than necessary. The last position > should be kept around so that we can compare. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4412) No need to retry scan operation on the same server in case of RegionServerStoppedException
[
https://issues.apache.org/jira/browse/HBASE-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118133#comment-13118133
]
Hudson commented on HBASE-4412:
---
Integrated in HBase-TRUNK #2272 (See
[https://builds.apache.org/job/HBase-TRUNK/2272/])
HBASE-4412 No need to retry scan operation on the same server in case of
RegionServerStoppedException (Ming Ma)
tedyu :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java
> No need to retry scan operation on the same server in case of
> RegionServerStoppedException
> --
>
> Key: HBASE-4412
> URL: https://issues.apache.org/jira/browse/HBASE-4412
> Project: HBase
> Issue Type: Bug
>Reporter: Ming Ma
>Assignee: Ming Ma
> Fix For: 0.92.0
>
> Attachments: HBASE-4412-trunk.patch
>
>
> ScannerCallable.java doesn't retry on the same server in case of
> NotServingRegionException.
> if (ioe instanceof NotServingRegionException) {
> // Throw a DNRE so that we break out of cycle of calling NSRE
> // when what we need is to open scanner against new location.
> // Attach NSRE to signal client that it needs to resetup scanner.
> throw new DoNotRetryIOException("Reset scanner", ioe);
> In the scenario when region server is in shutdown process,
> RegionServerStoppedException will be thrown. ScannerCallable still retry. It
> isn't necessary.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3130) [replication] ReplicationSource can't recover from session expired on remote clusters
[ https://issues.apache.org/jira/browse/HBASE-3130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118136#comment-13118136 ] Hudson commented on HBASE-3130: --- Integrated in HBase-TRUNK #2272 (See [https://builds.apache.org/job/HBase-TRUNK/2272/]) HBASE-3130 [replication] ReplicationSource can't recover from session expired on remote clusters (Chris Trezzo via JD) HBASE-4499 [replication] Source shouldn't update ZK if it didn't progress (Chris Trezzo via JD) jdcryans : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationPeer.java > [replication] ReplicationSource can't recover from session expired on remote > clusters > - > > Key: HBASE-3130 > URL: https://issues.apache.org/jira/browse/HBASE-3130 > Project: HBase > Issue Type: Bug > Components: replication >Affects Versions: 0.92.0 >Reporter: Jean-Daniel Cryans >Assignee: Chris Trezzo > Fix For: 0.92.0 > > Attachments: 3130-v2.txt, 3130-v3.txt, 3130-v4.txt, 3130.txt, > HBASE-3130-v5.patch > > > Currently ReplicationSource cannot recover when its zookeeper connection to > its remote cluster expires. HLogs are still being tracked, but a cluster > restart is required to continue replication (or a rolling restart). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4212) TestMasterFailover fails occasionally
[
https://issues.apache.org/jira/browse/HBASE-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118135#comment-13118135
]
Hudson commented on HBASE-4212:
---
Integrated in HBase-TRUNK #2272 (See
[https://builds.apache.org/job/HBase-TRUNK/2272/])
HBASE-4212 TestMasterFailover fails occasionally (Gao Jinchao)
tedyu :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
> TestMasterFailover fails occasionally
> -
>
> Key: HBASE-4212
> URL: https://issues.apache.org/jira/browse/HBASE-4212
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.90.4
>Reporter: gaojinchao
>Assignee: gaojinchao
> Fix For: 0.90.5
>
> Attachments: HBASE-4212_TrunkV1.patch, HBASE-4212_branch90V1.patch
>
>
> It seems a bug. The root in RIT can't be moved..
> In the failover process, it enforces root on-line. But not clean zk node.
> test will wait forever.
> void processFailover() throws KeeperException, IOException,
> InterruptedException {
>
> // we enforce on-line root.
> HServerInfo hsi =
>
> this.serverManager.getHServerInfo(this.catalogTracker.getMetaLocation());
> regionOnline(HRegionInfo.FIRST_META_REGIONINFO, hsi);
> hsi =
> this.serverManager.getHServerInfo(this.catalogTracker.getRootLocation());
> regionOnline(HRegionInfo.ROOT_REGIONINFO, hsi);
> It seems that we should wait finished as meta region
> int assignRootAndMeta()
> throws InterruptedException, IOException, KeeperException {
> int assigned = 0;
> long timeout = this.conf.getLong("hbase.catalog.verification.timeout",
> 1000);
> // Work on ROOT region. Is it in zk in transition?
> boolean rit = this.assignmentManager.
>
> processRegionInTransitionAndBlockUntilAssigned(HRegionInfo.ROOT_REGIONINFO);
> if (!catalogTracker.verifyRootRegionLocation(timeout)) {
> this.assignmentManager.assignRoot();
> this.catalogTracker.waitForRoot();
> //we need add this code and guarantee that the transition has completed
> this.assignmentManager.waitForAssignment(HRegionInfo.ROOT_REGIONINFO);
> assigned++;
> }
> logs:
> 2011-08-16 07:45:40,715 DEBUG
> [RegionServer:0;C4S2.site,47710,1313495126115-EventThread]
> zookeeper.ZooKeeperWatcher(252): regionserver:47710-0x131d2690f780004
> Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected,
> path=/hbase/unassigned/70236052
> 2011-08-16 07:45:40,715 DEBUG [RS_OPEN_ROOT-C4S2.site,47710,1313495126115-0]
> zookeeper.ZKAssign(712): regionserver:47710-0x131d2690f780004 Successfully
> transitioned node 70236052 from RS_ZK_REGION_OPENING to RS_ZK_REGION_OPENING
> 2011-08-16 07:45:40,715 DEBUG [Thread-760-EventThread]
> zookeeper.ZooKeeperWatcher(252): master:60701-0x131d2690f780009 Received
> ZooKeeper Event, type=NodeDataChanged, state=SyncConnected,
> path=/hbase/unassigned/70236052
> 2011-08-16 07:45:40,716 INFO [PostOpenDeployTasks:70236052]
> catalog.RootLocationEditor(62): Setting ROOT region location in ZooKeeper as
> C4S2.site:47710
> 2011-08-16 07:45:40,716 DEBUG [Thread-760-EventThread]
> zookeeper.ZKUtil(1109): master:60701-0x131d2690f780009 Retrieved 52 byte(s)
> of data from znode /hbase/unassigned/70236052 and set watcher;
> region=-ROOT-,,0, server=C4S2.site,47710,1313495126115,
> state=RS_ZK_REGION_OPENING
> 2011-08-16 07:45:40,717 DEBUG [Thread-760-EventThread]
> master.AssignmentManager(477): Handling transition=RS_ZK_REGION_OPENING,
> server=C4S2.site,47710,1313495126115, region=70236052/-ROOT-
> 2011-08-16 07:45:40,725 DEBUG [RS_OPEN_ROOT-C4S2.site,47710,1313495126115-0]
> zookeeper.ZKAssign(661): regionserver:47710-0x131d2690f780004 Attempting to
> transition node 70236052/-ROOT- from RS_ZK_REGION_OPENING to
> RS_ZK_REGION_OPENED
> 2011-08-16 07:45:40,727 DEBUG [RS_OPEN_ROOT-C4S2.site,47710,1313495126115-0]
> zookeeper.ZKUtil(1109): regionserver:47710-0x131d2690f780004 Retrieved 52
> byte(s) of data from znode /hbase/unassigned/70236052; data=region=-ROOT-,,0,
> server=C4S2.site,47710,1313495126115, state=RS_ZK_REGION_OPENING
> 2011-08-16 07:45:40,740 DEBUG
> [RegionServer:0;C4S2.site,47710,1313495126115-EventThread]
> zookeeper.ZooKeeperWatcher(252): regionserver:47710-0x131d2690f780004
> Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected,
> path=/hbase/unassigned/70236052
> 2011-08-16 07:45:40,740 DEBUG [Thread-760-EventThread]
> zookeeper.ZooKeeperWatcher(252): master:60701-0x131d2690f780009 Received
> ZooKeeper Event, type=NodeDataChanged, state=SyncConnected,
> path=/hbase/unassigned/70236052
> 2011-08-16 07:45:40,740 DEBUG [RS_OPEN_ROOT-C4S2.site,47710,1313495126115-0]
> zookeeper.ZKAssign(712): regionserver:47710-0x131d2690f780
[jira] [Commented] (HBASE-2794) Utilize ROWCOL bloom filter if multiple columns within same family are requested in a Get
[
https://issues.apache.org/jira/browse/HBASE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118507#comment-13118507
]
Hudson commented on HBASE-2794:
---
Integrated in HBase-0.92 #34 (See
[https://builds.apache.org/job/HBase-0.92/34/])
HBASE-2794 Utilize ROWCOL bloom filter if multiple columns within same
family
are requested in a Get (Mikhail Bautin)
tedyu :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/KeyValue.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/AbstractKeyValueScanner.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueHeap.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueScanner.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/CollectionBackedScanner.java
*
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestKeyValueHeap.java
*
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestMultiColumnScanner.java
*
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestScanWithBloomError.java
> Utilize ROWCOL bloom filter if multiple columns within same family are
> requested in a Get
> -
>
> Key: HBASE-2794
> URL: https://issues.apache.org/jira/browse/HBASE-2794
> Project: HBase
> Issue Type: Improvement
> Components: performance
>Reporter: Kannan Muthukkaruppan
>Assignee: Mikhail Bautin
> Fix For: 0.92.0
>
>
> Noticed the following snippet in StoreFile.java:Scanner:shouldSeek():
> {code}
> switch(bloomFilterType) {
> case ROW:
> key = row;
> break;
> case ROWCOL:
> if (columns.size() == 1) {
> byte[] col = columns.first();
> key = Bytes.add(row, col);
> break;
> }
> //$FALL-THROUGH$
> default:
> return true;
> }
> {code}
> If columns.size > 1, then we currently don't take advantage of the bloom
> filter. We should optimize this to check bloom for each of columns and if
> none of the columns are present in the bloom avoid opening the file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4476) Compactions must fail if column tracker gets columns out of order
[
https://issues.apache.org/jira/browse/HBASE-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118588#comment-13118588
]
Hudson commented on HBASE-4476:
---
Integrated in HBase-0.92 #35 (See
[https://builds.apache.org/job/HBase-0.92/35/])
HBASE-4476 Compactions must fail if column tracker gets columns out of order
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/ColumnTracker.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/ScanWildcardColumnTracker.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
*
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestExplicitColumnTracker.java
*
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestScanWildcardColumnTracker.java
> Compactions must fail if column tracker gets columns out of order
> -
>
> Key: HBASE-4476
> URL: https://issues.apache.org/jira/browse/HBASE-4476
> Project: HBase
> Issue Type: Bug
>Reporter: Mikhail Bautin
>Assignee: Mikhail Bautin
> Fix For: 0.92.0, 0.94.0
>
> Attachments: HBASE-4476_v2_2011-09-27_17_38_48.patch
>
>
> We found this in ScanWildcardColumnTracker:
> // new col < oldcol
> // if (cmp < 0) {
> // WARNING: This means that very likely an edit for some other family
> // was incorrectly stored into the store for this one. Continue, but
> // complain.
> LOG.error("ScanWildcardColumnTracker.checkColumn ran " +
> "into a column actually smaller than the previous column: " +
> This went under the radar in our dark launch cluster when a column family
> name was misspelled first, but then was "renamed" by renaming directories in
> the HBase storage directory tree. We ended up with inconsistent data, but
> compactions still succeeded most of the time, likely discarding part of input
> data.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4209) The HBase hbase-daemon.sh SIGKILLs master when stopping it
[ https://issues.apache.org/jira/browse/HBASE-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118587#comment-13118587 ] Hudson commented on HBASE-4209: --- Integrated in HBase-0.92 #35 (See [https://builds.apache.org/job/HBase-0.92/35/]) HBASE-4209 The HBase hbase-daemon.sh SIGKILLs master when stopping it stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/ShutdownHook.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java > The HBase hbase-daemon.sh SIGKILLs master when stopping it > -- > > Key: HBASE-4209 > URL: https://issues.apache.org/jira/browse/HBASE-4209 > Project: HBase > Issue Type: Bug > Components: master >Reporter: Roman Shaposhnik >Assignee: Roman Shaposhnik > Fix For: 0.92.0 > > Attachments: 4209-v3.txt, HBASE-4209.final.patch.txt, > HBASE-4209.patch.txt > > > There's a bit of code in hbase-daemon.sh that makes HBase master being > SIGKILLed when stopping it rather than trying SIGTERM (like it does for other > daemons). When HBase is executed in a standalone mode (and the only daemon > you need to run is master) that causes newly created tables to go missing as > unflushed data is thrown out. If there was not a good reason to kill master > with SIGKILL perhaps we can take that special case out and rely on SIGTERM. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4487) The increment operation can release the rowlock before sync-ing the Hlog
[ https://issues.apache.org/jira/browse/HBASE-4487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118630#comment-13118630 ] Hudson commented on HBASE-4487: --- Integrated in HBase-TRUNK #2274 (See [https://builds.apache.org/job/HBase-TRUNK/2274/]) HBASE-4487 The increment operation can release the rowlock before sync-ing the Hlog (dhruba borthakur) tedyu : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestIncrement.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogBench.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALActionsListener.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java > The increment operation can release the rowlock before sync-ing the Hlog > > > Key: HBASE-4487 > URL: https://issues.apache.org/jira/browse/HBASE-4487 > Project: HBase > Issue Type: Improvement >Reporter: dhruba borthakur >Assignee: dhruba borthakur > Fix For: 0.94.0 > > Attachments: 4487-v7.txt, appendNoSync4.txt, appendNoSync5.txt, > appendNoSync6.txt > > > This allows for better throughput when there are hot rows.I have seen this > change make a single row update improve from 400 increments/sec/server to > 4000 increments/sec/server. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4476) Compactions must fail if column tracker gets columns out of order
[
https://issues.apache.org/jira/browse/HBASE-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118631#comment-13118631
]
Hudson commented on HBASE-4476:
---
Integrated in HBase-TRUNK #2274 (See
[https://builds.apache.org/job/HBase-TRUNK/2274/])
HBASE-4476 Compactions must fail if column tracker gets columns out of order
stack :
Files :
* /hbase/trunk/CHANGES.txt
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ColumnTracker.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanWildcardColumnTracker.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestExplicitColumnTracker.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestScanWildcardColumnTracker.java
> Compactions must fail if column tracker gets columns out of order
> -
>
> Key: HBASE-4476
> URL: https://issues.apache.org/jira/browse/HBASE-4476
> Project: HBase
> Issue Type: Bug
>Reporter: Mikhail Bautin
>Assignee: Mikhail Bautin
> Fix For: 0.92.0, 0.94.0
>
> Attachments: HBASE-4476_v2_2011-09-27_17_38_48.patch
>
>
> We found this in ScanWildcardColumnTracker:
> // new col < oldcol
> // if (cmp < 0) {
> // WARNING: This means that very likely an edit for some other family
> // was incorrectly stored into the store for this one. Continue, but
> // complain.
> LOG.error("ScanWildcardColumnTracker.checkColumn ran " +
> "into a column actually smaller than the previous column: " +
> This went under the radar in our dark launch cluster when a column family
> name was misspelled first, but then was "renamed" by renaming directories in
> the HBase storage directory tree. We ended up with inconsistent data, but
> compactions still succeeded most of the time, likely discarding part of input
> data.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2794) Utilize ROWCOL bloom filter if multiple columns within same family are requested in a Get
[
https://issues.apache.org/jira/browse/HBASE-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118632#comment-13118632
]
Hudson commented on HBASE-2794:
---
Integrated in HBase-TRUNK #2274 (See
[https://builds.apache.org/job/HBase-TRUNK/2274/])
HBASE-2794 Utilize ROWCOL bloom filter if multiple columns within same
family
are requested in a Get (Mikhail Bautin)
tedyu :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/KeyValue.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/AbstractKeyValueScanner.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueHeap.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueScanner.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/CollectionBackedScanner.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestKeyValueHeap.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestMultiColumnScanner.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestScanWithBloomError.java
> Utilize ROWCOL bloom filter if multiple columns within same family are
> requested in a Get
> -
>
> Key: HBASE-2794
> URL: https://issues.apache.org/jira/browse/HBASE-2794
> Project: HBase
> Issue Type: Improvement
> Components: performance
>Reporter: Kannan Muthukkaruppan
>Assignee: Mikhail Bautin
> Fix For: 0.92.0
>
>
> Noticed the following snippet in StoreFile.java:Scanner:shouldSeek():
> {code}
> switch(bloomFilterType) {
> case ROW:
> key = row;
> break;
> case ROWCOL:
> if (columns.size() == 1) {
> byte[] col = columns.first();
> key = Bytes.add(row, col);
> break;
> }
> //$FALL-THROUGH$
> default:
> return true;
> }
> {code}
> If columns.size > 1, then we currently don't take advantage of the bloom
> filter. We should optimize this to check bloom for each of columns and if
> none of the columns are present in the bloom avoid opening the file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4209) The HBase hbase-daemon.sh SIGKILLs master when stopping it
[ https://issues.apache.org/jira/browse/HBASE-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118629#comment-13118629 ] Hudson commented on HBASE-4209: --- Integrated in HBase-TRUNK #2274 (See [https://builds.apache.org/job/HBase-TRUNK/2274/]) HBASE-4209 The HBase hbase-daemon.sh SIGKILLs master when stopping it stack : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ShutdownHook.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java > The HBase hbase-daemon.sh SIGKILLs master when stopping it > -- > > Key: HBASE-4209 > URL: https://issues.apache.org/jira/browse/HBASE-4209 > Project: HBase > Issue Type: Bug > Components: master >Reporter: Roman Shaposhnik >Assignee: Roman Shaposhnik > Fix For: 0.92.0 > > Attachments: 4209-v3.txt, HBASE-4209.final.patch.txt, > HBASE-4209.patch.txt > > > There's a bit of code in hbase-daemon.sh that makes HBase master being > SIGKILLed when stopping it rather than trying SIGTERM (like it does for other > daemons). When HBase is executed in a standalone mode (and the only daemon > you need to run is master) that causes newly created tables to go missing as > unflushed data is thrown out. If there was not a good reason to kill master > with SIGKILL perhaps we can take that special case out and rely on SIGTERM. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4515) User.getCurrent() can fail to initialize the current user
[
https://issues.apache.org/jira/browse/HBASE-4515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118657#comment-13118657
]
Hudson commented on HBASE-4515:
---
Integrated in HBase-TRUNK #2275 (See
[https://builds.apache.org/job/HBase-TRUNK/2275/])
HBASE-4515 User.getCurrent() can fail to initialize the current user
garyh :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/security/User.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/security/TestUser.java
> User.getCurrent() can fail to initialize the current user
> -
>
> Key: HBASE-4515
> URL: https://issues.apache.org/jira/browse/HBASE-4515
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.90.4, 0.92.0, 0.94.0
>Reporter: Jonathan Hsieh
>Assignee: Gary Helmling
> Fix For: 0.92.0, 0.94.0, 0.90.5
>
> Attachments: 0001-HBASE-4515-User.getCurrent-can-cause-NPE.patch,
> HBASE-4515_0.90_final.patch, HBASE-4515_0.92_final.patch,
> HBASE-4515_trunk.patch, HBASE-4515_trunk_final.patch
>
>
> When testing with miniclusters that shutdown and are restarted, sometimes a
> call to User.getCurrent().getName() NPEs when attempting to restart hbase.
> Oddly this happens consistently on particular branches and not on others. I
> don't know or understand why this happens but it has something to do with the
> getCurrentUGI call in o.a.h.h.security.User.HadoopUser sometimes returning
> null and sometimes returning data.
> {code}
>private HadoopUser() {
> try {
> ugi = (UserGroupInformation) callStatic("getCurrentUGI");
> if (ugi == null) {
> LOG.warn("Although successfully retrieved UserGroupInformation"
> + " it was null!");
> }
> } catch (RuntimeException re) {
> {code}
> This patch essentially is a workaround -- it propagates the null so that
> clients can check and avoid the NPE.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4517) Document new replication features in 0.92
[ https://issues.apache.org/jira/browse/HBASE-4517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118656#comment-13118656 ] Hudson commented on HBASE-4517: --- Integrated in HBase-TRUNK #2275 (See [https://builds.apache.org/job/HBase-TRUNK/2275/]) HBASE-4517 Document new replication features in 0.92 HBASE-4517 Document new replication features in 0.92 stack : Files : * /hbase/trunk/src/main/javadoc/org/apache/hadoop/hbase/replication/package.html stack : Files : * /hbase/trunk/src/site/xdoc/replication.xml > Document new replication features in 0.92 > - > > Key: HBASE-4517 > URL: https://issues.apache.org/jira/browse/HBASE-4517 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 0.92.0, 0.94.0 > > Attachments: 4517-v2.txt, 4517.txt > > > Document changes from HBASE-2195 and HBASE-2196 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4506) [hbck] Allow HBaseFsck to be instantiated without connecting
[ https://issues.apache.org/jira/browse/HBASE-4506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118701#comment-13118701 ] Hudson commented on HBASE-4506: --- Integrated in HBase-TRUNK #2276 (See [https://builds.apache.org/job/HBase-TRUNK/2276/]) HBASE-4506 [hbck] Allow HBaseFsck to be instantiated without connecting HBASE-4506 [hbck] Allow HBaseFsck to be instantiated without connecting stack : Files : * /hbase/trunk/CHANGES.txt stack : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java > [hbck] Allow HBaseFsck to be instantiated without connecting > > > Key: HBASE-4506 > URL: https://issues.apache.org/jira/browse/HBASE-4506 > Project: HBase > Issue Type: Improvement > Components: hbck >Affects Versions: 0.90.5 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 0.90.5 > > Attachments: > 0001-HBASE-4506-hbck-Allow-HBaseFsck-to-be-instantiated-w.patch, > hbase-4506-0.90.patch > > > This is a semantics preserving patch that allows for offline meta rebuild > (HBASE-4377) to reuse code in the existing hbck code when hbase is down. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4509) [hbck] Improve region map output
[ https://issues.apache.org/jira/browse/HBASE-4509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118702#comment-13118702 ] Hudson commented on HBASE-4509: --- Integrated in HBase-TRUNK #2276 (See [https://builds.apache.org/job/HBase-TRUNK/2276/]) HBASE-4509 [hbck] Improve region map output stack : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java > [hbck] Improve region map output > > > Key: HBASE-4509 > URL: https://issues.apache.org/jira/browse/HBASE-4509 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.0, 0.94.0, 0.90.5 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 0.90.5 > > Attachments: 0001-HBASE-4509-hbck-Improve-region-map-output.patch, > hbase-4509-pre4506.patch > > > HBASE-4375 added a region coverage visualization to hbck in details mode. > When users have binary row keys the output is difficult to parse (awk/sed) or > pull into programs (numeric, excel) capable of handling tsv formatted data. > This patch > * improves output by using Bytes.toStringBinary (which escapes binary) > instead of Bytes.toString when printing keys, > * suggests some repair actions, and > * collects "problem group" that groups regions that are overlapping. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4517) Document new replication features in 0.92
[ https://issues.apache.org/jira/browse/HBASE-4517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119087#comment-13119087 ] Hudson commented on HBASE-4517: --- Integrated in HBase-0.92 #37 (See [https://builds.apache.org/job/HBase-0.92/37/]) HBASE-4517 Document new replication features in 0.92 stack : Files : * /hbase/branches/0.92/src/main/javadoc/org/apache/hadoop/hbase/replication/package.html > Document new replication features in 0.92 > - > > Key: HBASE-4517 > URL: https://issues.apache.org/jira/browse/HBASE-4517 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 0.92.0, 0.94.0 > > Attachments: 4517-v2.txt, 4517.txt > > > Document changes from HBASE-2195 and HBASE-2196 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4515) User.getCurrent() can fail to initialize the current user
[
https://issues.apache.org/jira/browse/HBASE-4515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119088#comment-13119088
]
Hudson commented on HBASE-4515:
---
Integrated in HBase-0.92 #37 (See
[https://builds.apache.org/job/HBase-0.92/37/])
HBASE-4515 User.getCurrent() can fail to initialize the current user
garyh :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/security/User.java
*
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/security/TestUser.java
> User.getCurrent() can fail to initialize the current user
> -
>
> Key: HBASE-4515
> URL: https://issues.apache.org/jira/browse/HBASE-4515
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.90.4, 0.92.0, 0.94.0
>Reporter: Jonathan Hsieh
>Assignee: Gary Helmling
> Fix For: 0.92.0, 0.94.0, 0.90.5
>
> Attachments: 0001-HBASE-4515-User.getCurrent-can-cause-NPE.patch,
> HBASE-4515_0.90_final.patch, HBASE-4515_0.92_final.patch,
> HBASE-4515_trunk.patch, HBASE-4515_trunk_final.patch
>
>
> When testing with miniclusters that shutdown and are restarted, sometimes a
> call to User.getCurrent().getName() NPEs when attempting to restart hbase.
> Oddly this happens consistently on particular branches and not on others. I
> don't know or understand why this happens but it has something to do with the
> getCurrentUGI call in o.a.h.h.security.User.HadoopUser sometimes returning
> null and sometimes returning data.
> {code}
>private HadoopUser() {
> try {
> ugi = (UserGroupInformation) callStatic("getCurrentUGI");
> if (ugi == null) {
> LOG.warn("Although successfully retrieved UserGroupInformation"
> + " it was null!");
> }
> } catch (RuntimeException re) {
> {code}
> This patch essentially is a workaround -- it propagates the null so that
> clients can check and avoid the NPE.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4506) [hbck] Allow HBaseFsck to be instantiated without connecting
[ https://issues.apache.org/jira/browse/HBASE-4506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119089#comment-13119089 ] Hudson commented on HBASE-4506: --- Integrated in HBase-0.92 #37 (See [https://builds.apache.org/job/HBase-0.92/37/]) HBASE-4506 [hbck] Allow HBaseFsck to be instantiated without connecting HBASE-4506 [hbck] Allow HBaseFsck to be instantiated without connecting stack : Files : * /hbase/branches/0.92/CHANGES.txt stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java > [hbck] Allow HBaseFsck to be instantiated without connecting > > > Key: HBASE-4506 > URL: https://issues.apache.org/jira/browse/HBASE-4506 > Project: HBase > Issue Type: Improvement > Components: hbck >Affects Versions: 0.90.5 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 0.90.5 > > Attachments: > 0001-HBASE-4506-hbck-Allow-HBaseFsck-to-be-instantiated-w.patch, > hbase-4506-0.90.patch > > > This is a semantics preserving patch that allows for offline meta rebuild > (HBASE-4377) to reuse code in the existing hbck code when hbase is down. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4509) [hbck] Improve region map output
[ https://issues.apache.org/jira/browse/HBASE-4509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119090#comment-13119090 ] Hudson commented on HBASE-4509: --- Integrated in HBase-0.92 #37 (See [https://builds.apache.org/job/HBase-0.92/37/]) HBASE-4509 [hbck] Improve region map output stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java > [hbck] Improve region map output > > > Key: HBASE-4509 > URL: https://issues.apache.org/jira/browse/HBASE-4509 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.0, 0.94.0, 0.90.5 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 0.90.5 > > Attachments: 0001-HBASE-4509-hbck-Improve-region-map-output.patch, > hbase-4509-pre4506.patch > > > HBASE-4375 added a region coverage visualization to hbck in details mode. > When users have binary row keys the output is difficult to parse (awk/sed) or > pull into programs (numeric, excel) capable of handling tsv formatted data. > This patch > * improves output by using Bytes.toStringBinary (which escapes binary) > instead of Bytes.toString when printing keys, > * suggests some repair actions, and > * collects "problem group" that groups regions that are overlapping. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4496) HFile V2 does not honor setCacheBlocks when scanning.
[ https://issues.apache.org/jira/browse/HBASE-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119086#comment-13119086 ] Hudson commented on HBASE-4496: --- Integrated in HBase-0.92 #37 (See [https://builds.apache.org/job/HBase-0.92/37/]) HBASE-4496 HFile V2 does not honor setCacheBlocks when scanning (Lars and Mikhail) tedyu : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockIndex.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java > HFile V2 does not honor setCacheBlocks when scanning. > - > > Key: HBASE-4496 > URL: https://issues.apache.org/jira/browse/HBASE-4496 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.92.0, 0.94.0 >Reporter: Lars Hofhansl >Assignee: Mikhail Bautin > Fix For: 0.92.0, 0.94.0 > > Attachments: 4496.final, 4496.txt > > > While testing the LRU cache during the scanning I noticed quite some churn in > the cache even when Scan.cacheBlocks is set to false. After debugging this, I > found that HFile V2 always caches blocks in the LRU cache regardless of the > cacheBlocks setting. > Here's a trace (from Eclipse) showing the problem: > HFileReaderV2.readBlock(long, int, boolean, boolean, boolean) line: 279 > HFileReaderV2.readBlockData(long, long, int, boolean) line: 219 > HFileBlockIndex$BlockIndexReader.seekToDataBlock(byte[], int, int, > HFileBlock) line: 191 > HFileReaderV2$ScannerV2.seekTo(byte[], int, int, boolean) line: 502 > HFileReaderV2$ScannerV2.reseekTo(byte[], int, int) line: 539 > StoreFileScanner.reseekAtOrAfter(HFileScanner, KeyValue) line: 151 > StoreFileScanner.reseek(KeyValue) line: 110 > KeyValueHeap.reseek(KeyValue) line: 255 > StoreScanner.reseek(KeyValue) line: 409 > StoreScanner.next(List, int) line: 304 > KeyValueHeap.next(List, int) line: 114 > KeyValueHeap.next(List) line: 143 > HRegion$RegionScannerImpl.nextRow(byte[]) line: 2774 > HRegion$RegionScannerImpl.nextInternal(int) line: 2722 > HRegion$RegionScannerImpl.next(List, int) line: 2682 > HRegion$RegionScannerImpl.next(List) line: 2699 > HRegionServer.next(long, int) line: 2092 > Every scanner.next causes a reseek, which eventually causes a call to > HFileBlockIndex$BlockIndexReader.seekToDataBlock(...) at which point the > cacheBlocks information is lost. HFileReaderV2.readBlockData calls > HFileReaderV2.readBlock with cacheBlocks set unconditionally to true. > The fix is not immediately clear, unless we want to pass cacheBlocks to > HFileBlockIndex$BlockIndexReader.seekToDataBlock and then on to > HFileBlock.BasicReader.readBlockData and all its implementers, which is ugly > as readBlockData should not care about caching. > Avoiding caching during scans is somewhat important for us. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4496) HFile V2 does not honor setCacheBlocks when scanning.
[ https://issues.apache.org/jira/browse/HBASE-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119108#comment-13119108 ] Hudson commented on HBASE-4496: --- Integrated in HBase-0.92 #38 (See [https://builds.apache.org/job/HBase-0.92/38/]) HBASE-4496 Addendum for TestBlocksRead tedyu : Files : * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java > HFile V2 does not honor setCacheBlocks when scanning. > - > > Key: HBASE-4496 > URL: https://issues.apache.org/jira/browse/HBASE-4496 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.92.0, 0.94.0 >Reporter: Lars Hofhansl >Assignee: Mikhail Bautin > Fix For: 0.92.0, 0.94.0 > > Attachments: 4496.addendum, 4496.final, 4496.txt > > > While testing the LRU cache during the scanning I noticed quite some churn in > the cache even when Scan.cacheBlocks is set to false. After debugging this, I > found that HFile V2 always caches blocks in the LRU cache regardless of the > cacheBlocks setting. > Here's a trace (from Eclipse) showing the problem: > HFileReaderV2.readBlock(long, int, boolean, boolean, boolean) line: 279 > HFileReaderV2.readBlockData(long, long, int, boolean) line: 219 > HFileBlockIndex$BlockIndexReader.seekToDataBlock(byte[], int, int, > HFileBlock) line: 191 > HFileReaderV2$ScannerV2.seekTo(byte[], int, int, boolean) line: 502 > HFileReaderV2$ScannerV2.reseekTo(byte[], int, int) line: 539 > StoreFileScanner.reseekAtOrAfter(HFileScanner, KeyValue) line: 151 > StoreFileScanner.reseek(KeyValue) line: 110 > KeyValueHeap.reseek(KeyValue) line: 255 > StoreScanner.reseek(KeyValue) line: 409 > StoreScanner.next(List, int) line: 304 > KeyValueHeap.next(List, int) line: 114 > KeyValueHeap.next(List) line: 143 > HRegion$RegionScannerImpl.nextRow(byte[]) line: 2774 > HRegion$RegionScannerImpl.nextInternal(int) line: 2722 > HRegion$RegionScannerImpl.next(List, int) line: 2682 > HRegion$RegionScannerImpl.next(List) line: 2699 > HRegionServer.next(long, int) line: 2092 > Every scanner.next causes a reseek, which eventually causes a call to > HFileBlockIndex$BlockIndexReader.seekToDataBlock(...) at which point the > cacheBlocks information is lost. HFileReaderV2.readBlockData calls > HFileReaderV2.readBlock with cacheBlocks set unconditionally to true. > The fix is not immediately clear, unless we want to pass cacheBlocks to > HFileBlockIndex$BlockIndexReader.seekToDataBlock and then on to > HFileBlock.BasicReader.readBlockData and all its implementers, which is ugly > as readBlockData should not care about caching. > Avoiding caching during scans is somewhat important for us. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4526) special case for stopping master in hbase-daemon.sh is no longer required
[
https://issues.apache.org/jira/browse/HBASE-4526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119515#comment-13119515
]
Hudson commented on HBASE-4526:
---
Integrated in HBase-0.92 #39 (See
[https://builds.apache.org/job/HBase-0.92/39/])
HBASE-4526 special case for stopping master in hbase-daemon.sh is no longer
required
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/bin/hbase-daemon.sh
> special case for stopping master in hbase-daemon.sh is no longer required
> -
>
> Key: HBASE-4526
> URL: https://issues.apache.org/jira/browse/HBASE-4526
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Affects Versions: 0.90.4
>Reporter: Roman Shaposhnik
>Assignee: Roman Shaposhnik
>Priority: Minor
> Fix For: 0.92.0
>
> Attachments: HBASE-4526.patch.txt
>
>
> Now that HBASE-4209 is finally done (many thanks to stack for help and
> encouragement!) I don't think there's any reason to keep this bit of code in
> hbase-daemon.sh:
> {noformat}
> if [ "$command" = "master" ]; then
> echo "`date` Killing $command" >> $loglog
> kill -9 `cat $pid` > /dev/null 2>&1
> else
> echo "`date` Killing $command" >> $loglog
> kill `cat $pid` > /dev/null 2>&1
> fi
> {noformat}
> I suggest we remove the special case completely, since I don't think it
> serves any useful purpose (patch attached).
> Now, as an additional precautionary measure. We can try to follow up with a
> SIGKILL once a certain timeout expires.
> Let me know if you think it is necessary to do so and I'll update the patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4454) Add failsafe plugin to build and rename integration tests
[ https://issues.apache.org/jira/browse/HBASE-4454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119715#comment-13119715 ] Hudson commented on HBASE-4454: --- Integrated in HBase-0.92 #40 (See [https://builds.apache.org/job/HBase-0.92/40/]) HBASE-4531 hbase-4454 failsafe broke mvn site; back it out or fix > Add failsafe plugin to build and rename integration tests > - > > Key: HBASE-4454 > URL: https://issues.apache.org/jira/browse/HBASE-4454 > Project: HBase > Issue Type: Improvement >Reporter: Jesse Yates > Fix For: 0.92.0 > > Attachments: mvn_HBASE-4454.patch > > > Add the maven-failsafe-plugin to the build process so we can run integration > tests with "mvn verify". This will also involve a renaming of integration > tests to conform to a new integration test regex. > This is a stopgap measure while we until break them out into their own module. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4531) hbase-4454 failsafe broke mvn site; back it out or fix
[
https://issues.apache.org/jira/browse/HBASE-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119714#comment-13119714
]
Hudson commented on HBASE-4531:
---
Integrated in HBase-0.92 #40 (See
[https://builds.apache.org/job/HBase-0.92/40/])
HBASE-4531 hbase-4454 failsafe broke mvn site; back it out or fix
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/pom.xml
> hbase-4454 failsafe broke mvn site; back it out or fix
> --
>
> Key: HBASE-4531
> URL: https://issues.apache.org/jira/browse/HBASE-4531
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Akash Ashok
>Priority: Blocker
> Fix For: 0.92.0
>
> Attachments: HBase-4531-v2.patch, HBase-4531-v2.patch,
> HBase-4531.patch
>
>
> mvn site is broke in head of trunk. If I back out the last pom change it
> works again:
> {code}
>
> r1177168 | stack | 2011-09-29 05:42:13 + (Thu, 29 Sep 2011) | 1 line
> HBASE-4454 Add failsafe plugin to build and rename integration tests
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4473) NPE when executors are down but events are still coming in
[
https://issues.apache.org/jira/browse/HBASE-4473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119713#comment-13119713
]
Hudson commented on HBASE-4473:
---
Integrated in HBase-0.92 #40 (See
[https://builds.apache.org/job/HBase-0.92/40/])
HBASE-4473 NPE when executors are down but events are still coming in
jdcryans :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/executor/ExecutorService.java
*
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/executor/TestExecutorService.java
> NPE when executors are down but events are still coming in
> --
>
> Key: HBASE-4473
> URL: https://issues.apache.org/jira/browse/HBASE-4473
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.90.4
>Reporter: Jean-Daniel Cryans
>Assignee: Jean-Daniel Cryans
>Priority: Minor
> Fix For: 0.90.5
>
> Attachments: HBASE-4473-0.90.patch, HBASE-4473-v2.patch,
> HBASE-4473.patch
>
>
> Minor annoyance when shutting down a cluster and the master is still
> receiving events from Zookeeper:
> {quote}
> 2011-09-22 23:53:01,552 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher:
> master:6-0x3292d87deb004f Received InterruptedException, doing nothing
> here
> java.lang.InterruptedException
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:485)
> at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1317)
> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:726)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:938)
> at
> org.apache.hadoop.hbase.zookeeper.ZKAssign.deleteNode(ZKAssign.java:407)
> at
> org.apache.hadoop.hbase.zookeeper.ZKAssign.deleteOpenedNode(ZKAssign.java:284)
> at
> org.apache.hadoop.hbase.master.handler.OpenedRegionHandler.process(OpenedRegionHandler.java:88)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:156)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> ...
> 2011-09-22 23:53:01,558 DEBUG
> org.apache.hadoop.hbase.executor.ExecutorService: Executor service
> [MASTER_OPEN_REGION-sv2borg170:6] not found in {}
> 2011-09-22 23:53:01,558 ERROR org.apache.zookeeper.ClientCnxn: Error while
> calling watcher
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.executor.ExecutorService.submit(ExecutorService.java:220)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.handleRegion(AssignmentManager.java:447)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.nodeDataChanged(AssignmentManager.java:546)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:281)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
> {quote}
> It's annoying because it then spams you with a bunch of NPEs that have
> nothing to do with the reason the Master is shutting down. Googling I saw
> someone also had that issue in June: http://pastebin.com/5Tqrj0nq
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4334) HRegion.get never validates row
[ https://issues.apache.org/jira/browse/HBASE-4334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119741#comment-13119741 ] Hudson commented on HBASE-4334: --- Integrated in HBase-0.92 #41 (See [https://builds.apache.org/job/HBase-0.92/41/]) HBASE-4334 HRegion.get never validates row (Lars H) tedyu : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java > HRegion.get never validates row > --- > > Key: HBASE-4334 > URL: https://issues.apache.org/jira/browse/HBASE-4334 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.90.4 >Reporter: Joe Pallas >Assignee: Lars Hofhansl > Fix For: 0.92.0 > > Attachments: 4334-v2.txt, 4334-v3.txt, 4334.txt > > > If a client gets confused (possibly by a hole in .META., see HBASE-4333), it > may send a request to the wrong region. Paths through put, delete, > incrementColumnValue, and checkAndMutate all call checkRow either directly or > indirectly (through getLock). But get apparently does not. This can result > in returning an incorrect empty result instead of a WrongRegionException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4531) hbase-4454 failsafe broke mvn site; back it out or fix
[
https://issues.apache.org/jira/browse/HBASE-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120496#comment-13120496
]
Hudson commented on HBASE-4531:
---
Integrated in HBase-0.92 #42 (See
[https://builds.apache.org/job/HBase-0.92/42/])
HBASE-4531 hbase-4454 failsafe broke mvn site; back it out or fix
stack :
Files :
* /hbase/branches/0.92/pom.xml
> hbase-4454 failsafe broke mvn site; back it out or fix
> --
>
> Key: HBASE-4531
> URL: https://issues.apache.org/jira/browse/HBASE-4531
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Akash Ashok
>Priority: Blocker
> Fix For: 0.92.0
>
> Attachments: HBase-4531-v2.patch, HBase-4531-v2.patch,
> HBase-4531-v3.patch, HBase-4531.patch
>
>
> mvn site is broke in head of trunk. If I back out the last pom change it
> works again:
> {code}
>
> r1177168 | stack | 2011-09-29 05:42:13 + (Thu, 29 Sep 2011) | 1 line
> HBASE-4454 Add failsafe plugin to build and rename integration tests
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4520) Better handling of Bloom filter type discrepancy between HFile and CF config
[ https://issues.apache.org/jira/browse/HBASE-4520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120498#comment-13120498 ] Hudson commented on HBASE-4520: --- Integrated in HBase-0.92 #42 (See [https://builds.apache.org/job/HBase-0.92/42/]) HBASE-4520 Better handling of Bloom filter type discrepancy between HFile and CF config stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java > Better handling of Bloom filter type discrepancy between HFile and CF config > > > Key: HBASE-4520 > URL: https://issues.apache.org/jira/browse/HBASE-4520 > Project: HBase > Issue Type: Improvement >Reporter: Mikhail Bautin >Assignee: Mikhail Bautin >Priority: Minor > Fix For: 0.92.0 > > Attachments: > HBASE-4520_Better_handling_of_Bloom_filt-20111002010126-16a0d7ba.patch > > > Modify StoreFile to make it clear where Bloom filter type settings come from. > We have two sources of truth: (1) HFile; and (2) CF configuration. (1) takes > precedence in the reader, and (2) takes precedence in the writer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4454) Add failsafe plugin to build and rename integration tests
[ https://issues.apache.org/jira/browse/HBASE-4454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120497#comment-13120497 ] Hudson commented on HBASE-4454: --- Integrated in HBase-0.92 #42 (See [https://builds.apache.org/job/HBase-0.92/42/]) HBASE-4531 hbase-4454 failsafe broke mvn site; back it out or fix > Add failsafe plugin to build and rename integration tests > - > > Key: HBASE-4454 > URL: https://issues.apache.org/jira/browse/HBASE-4454 > Project: HBase > Issue Type: Improvement >Reporter: Jesse Yates > Fix For: 0.92.0 > > Attachments: mvn_HBASE-4454.patch > > > Add the maven-failsafe-plugin to the build process so we can run integration > tests with "mvn verify". This will also involve a renaming of integration > tests to conform to a new integration test regex. > This is a stopgap measure while we until break them out into their own module. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4537) Bad imports in TestUser from HBASE-4515 change
[
https://issues.apache.org/jira/browse/HBASE-4537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120631#comment-13120631
]
Hudson commented on HBASE-4537:
---
Integrated in HBase-0.92 #43 (See
[https://builds.apache.org/job/HBase-0.92/43/])
HBASE-4537 Unused TestUser imports break compilation against secure Hadoop
garyh :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/security/TestUser.java
> Bad imports in TestUser from HBASE-4515 change
> --
>
> Key: HBASE-4537
> URL: https://issues.apache.org/jira/browse/HBASE-4537
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0, 0.94.0, 0.90.5
>Reporter: Gary Helmling
>Assignee: Gary Helmling
> Fix For: 0.90.5
>
> Attachments: HBASE-4537.patch
>
>
> The HBASE-4515 patch mistakenly added unused imports to TestUser:
> {code}
> import org.apache.hadoop.security.UnixUserGroupInformation;
> import org.apache.hadoop.security.UserGroupInformation;
> {code}
> Both should be removed. UnixUserGroupInformation is specific to non-secure
> Hadoop and importing it will break compilation against secure Hadoop variants
> (0.20.205, 0.22, 0.23).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4479) TestMasterFailover failure in Hbase-0.92#17
[ https://issues.apache.org/jira/browse/HBASE-4479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120724#comment-13120724 ] Hudson commented on HBASE-4479: --- Integrated in HBase-0.92 #44 (See [https://builds.apache.org/job/HBase-0.92/44/]) HBASE-4479 TestMasterFailover failure in Hbase-0.92#17 (Ram) ramkrishna : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java > TestMasterFailover failure in Hbase-0.92#17 > --- > > Key: HBASE-4479 > URL: https://issues.apache.org/jira/browse/HBASE-4479 > Project: HBase > Issue Type: Bug >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 0.92.0 > > Attachments: HBASE-4479.patch > > > When the master restarted it was not able to get any servers online and the > restart was a clean restart. > Hence there were no regions to assign. > Hence the retainAssignment tries to get one of the regions and uses > RANDOM.getInt(size). Here size is 0. > So ideally 0 is not accepted here. Hence we have got an exception making the > master not to come up and the test case timeout. > Though we need to see if really no regions was expected when the master came > up, but this JIRA's intent is to deal such scenario where the size can be 0. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4494) AvroServer:: get fails with NPE on a non-existent row
[ https://issues.apache.org/jira/browse/HBASE-4494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120725#comment-13120725 ] Hudson commented on HBASE-4494: --- Integrated in HBase-0.92 #44 (See [https://builds.apache.org/job/HBase-0.92/44/]) HBASE-4494 AvroServer:: get fails with NPE on a non-existent row stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/avro/AvroUtil.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/avro/TestAvroUtil.java > AvroServer:: get fails with NPE on a non-existent row > - > > Key: HBASE-4494 > URL: https://issues.apache.org/jira/browse/HBASE-4494 > Project: HBase > Issue Type: Bug > Components: avro >Affects Versions: 0.90.4 >Reporter: Kay Kay >Assignee: Kay Kay > Fix For: 0.92.0 > > Attachments: HBASE-4494.patch > > > Try to submit a get request to the avro gateway. > If the row specified for a given table does not exist, the server request > fails with a NPE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4494) AvroServer:: get fails with NPE on a non-existent row
[ https://issues.apache.org/jira/browse/HBASE-4494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120770#comment-13120770 ] Hudson commented on HBASE-4494: --- Integrated in HBase-TRUNK #2295 (See [https://builds.apache.org/job/HBase-TRUNK/2295/]) HBASE-4494 AvroServer:: get fails with NPE on a non-existent row stack : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/avro/AvroUtil.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/avro/TestAvroUtil.java > AvroServer:: get fails with NPE on a non-existent row > - > > Key: HBASE-4494 > URL: https://issues.apache.org/jira/browse/HBASE-4494 > Project: HBase > Issue Type: Bug > Components: avro >Affects Versions: 0.90.4 >Reporter: Kay Kay >Assignee: Kay Kay > Fix For: 0.92.0 > > Attachments: HBASE-4494.patch > > > Try to submit a get request to the avro gateway. > If the row specified for a given table does not exist, the server request > fails with a NPE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4479) TestMasterFailover failure in Hbase-0.92#17
[ https://issues.apache.org/jira/browse/HBASE-4479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120769#comment-13120769 ] Hudson commented on HBASE-4479: --- Integrated in HBase-TRUNK #2295 (See [https://builds.apache.org/job/HBase-TRUNK/2295/]) HBASE-4479 TestMasterFailover failure in Hbase-0.92#17(Ram) ramkrishna : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java > TestMasterFailover failure in Hbase-0.92#17 > --- > > Key: HBASE-4479 > URL: https://issues.apache.org/jira/browse/HBASE-4479 > Project: HBase > Issue Type: Bug >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 0.92.0 > > Attachments: HBASE-4479.patch > > > When the master restarted it was not able to get any servers online and the > restart was a clean restart. > Hence there were no regions to assign. > Hence the retainAssignment tries to get one of the regions and uses > RANDOM.getInt(size). Here size is 0. > So ideally 0 is not accepted here. Hence we have got an exception making the > master not to come up and the test case timeout. > Though we need to see if really no regions was expected when the master came > up, but this JIRA's intent is to deal such scenario where the size can be 0. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4481) TestMergeTool failed in 0.92 build 20
[
https://issues.apache.org/jira/browse/HBASE-4481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121312#comment-13121312
]
Hudson commented on HBASE-4481:
---
Integrated in HBase-0.92 #45 (See
[https://builds.apache.org/job/HBase-0.92/45/])
HBASE-4481 TestMergeTool failed in 0.92 build 20
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestMergeTool.java
> TestMergeTool failed in 0.92 build 20
> -
>
> Key: HBASE-4481
> URL: https://issues.apache.org/jira/browse/HBASE-4481
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: Ted Yu
>Priority: Critical
> Attachments: 4481-dbg.txt, 4481.txt
>
>
> TestMergeTool failed due to the following exception:
> {code}
> java.lang.StringIndexOutOfBoundsException: String index out of range: -1
> at java.lang.String.substring(String.java:1937)
> at org.apache.hadoop.hbase.ServerName.parseHostname(ServerName.java:81)
> at org.apache.hadoop.hbase.ServerName.(ServerName.java:63)
> at
> org.apache.hadoop.hbase.MasterAddressTracker.getMasterAddress(MasterAddressTracker.java:62)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:583)
> at org.apache.hadoop.hbase.client.HBaseAdmin.(HBaseAdmin.java:108)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:1588)
> at org.apache.hadoop.hbase.util.Merge.run(Merge.java:94)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at
> org.apache.hadoop.hbase.util.TestMergeTool.mergeAndVerify(TestMergeTool.java:186)
> at
> org.apache.hadoop.hbase.util.TestMergeTool.testMergeTool(TestMergeTool.java:264)
> {code}
> Log can be found at:
> https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.92/20/testReport/junit/org.apache.hadoop.hbase.util/TestMergeTool/testMergeTool/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4481) TestMergeTool failed in 0.92 build 20
[
https://issues.apache.org/jira/browse/HBASE-4481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121445#comment-13121445
]
Hudson commented on HBASE-4481:
---
Integrated in HBase-TRUNK #2297 (See
[https://builds.apache.org/job/HBase-TRUNK/2297/])
HBASE-4481 TestMergeTool failed in 0.92 build 20
stack :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestMergeTool.java
> TestMergeTool failed in 0.92 build 20
> -
>
> Key: HBASE-4481
> URL: https://issues.apache.org/jira/browse/HBASE-4481
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: Ted Yu
>Priority: Critical
> Attachments: 4481-dbg.txt, 4481.txt
>
>
> TestMergeTool failed due to the following exception:
> {code}
> java.lang.StringIndexOutOfBoundsException: String index out of range: -1
> at java.lang.String.substring(String.java:1937)
> at org.apache.hadoop.hbase.ServerName.parseHostname(ServerName.java:81)
> at org.apache.hadoop.hbase.ServerName.(ServerName.java:63)
> at
> org.apache.hadoop.hbase.MasterAddressTracker.getMasterAddress(MasterAddressTracker.java:62)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:583)
> at org.apache.hadoop.hbase.client.HBaseAdmin.(HBaseAdmin.java:108)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:1588)
> at org.apache.hadoop.hbase.util.Merge.run(Merge.java:94)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at
> org.apache.hadoop.hbase.util.TestMergeTool.mergeAndVerify(TestMergeTool.java:186)
> at
> org.apache.hadoop.hbase.util.TestMergeTool.testMergeTool(TestMergeTool.java:264)
> {code}
> Log can be found at:
> https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.92/20/testReport/junit/org.apache.hadoop.hbase.util/TestMergeTool/testMergeTool/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4501) [replication] Shutting down a stream leaves recovered sources running
[
https://issues.apache.org/jira/browse/HBASE-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121444#comment-13121444
]
Hudson commented on HBASE-4501:
---
Integrated in HBase-TRUNK #2297 (See
[https://builds.apache.org/job/HBase-TRUNK/2297/])
HBASE-4501 [replication] Shutting down a stream leaves recovered
sources running
jdcryans :
Files :
* /hbase/trunk/CHANGES.txt
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
> [replication] Shutting down a stream leaves recovered sources running
> -
>
> Key: HBASE-4501
> URL: https://issues.apache.org/jira/browse/HBASE-4501
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.90.4
>Reporter: Jean-Daniel Cryans
>Assignee: Jean-Daniel Cryans
> Fix For: 0.90.5
>
> Attachments: HBASE-4501.patch
>
>
> When removing a peer it will call ReplicationSourceManager.removePeer which
> calls closeRecoveredQueue which does this:
> {code}
> LOG.info("Done with the recovered queue " + src.getPeerClusterZnode());
> this.oldsources.remove(src);
> this.zkHelper.deleteSource(src.getPeerClusterZnode(), false);
> {code}
> This works in the case where the recovered source is done and is calling this
> method, but when removing a peer it never calls terminate on thus it leaving
> it running.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4541) [book] book.xml - claritying schema design section on versions
[ https://issues.apache.org/jira/browse/HBASE-4541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121446#comment-13121446 ] Hudson commented on HBASE-4541: --- Integrated in HBase-TRUNK #2297 (See [https://builds.apache.org/job/HBase-TRUNK/2297/]) HBASE-4541 clarifying versions in Schema Design dmeil : Files : * /hbase/trunk/src/docbkx/book.xml > [book] book.xml - claritying schema design section on versions > -- > > Key: HBASE-4541 > URL: https://issues.apache.org/jira/browse/HBASE-4541 > Project: HBase > Issue Type: Improvement >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Minor > Attachments: book_HBASE_4541.xml.patch > > > In the Schema Design chapter there was a section on versions and it wasn't > clearly differentiating between max and min versions. > * Moving the existing section on versions to a subsection called "Maximum > Number of Versions" so it's at the same level as "Minimum Number of > Versions", and added the "max" qualifier in that sub-section. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4501) [replication] Shutting down a stream leaves recovered sources running
[
https://issues.apache.org/jira/browse/HBASE-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121564#comment-13121564
]
Hudson commented on HBASE-4501:
---
Integrated in HBase-0.92 #46 (See
[https://builds.apache.org/job/HBase-0.92/46/])
HBASE-4501 [replication] Shutting down a stream leaves recovered
sources running
jdcryans :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
> [replication] Shutting down a stream leaves recovered sources running
> -
>
> Key: HBASE-4501
> URL: https://issues.apache.org/jira/browse/HBASE-4501
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.90.4
>Reporter: Jean-Daniel Cryans
>Assignee: Jean-Daniel Cryans
> Fix For: 0.90.5
>
> Attachments: HBASE-4501.patch
>
>
> When removing a peer it will call ReplicationSourceManager.removePeer which
> calls closeRecoveredQueue which does this:
> {code}
> LOG.info("Done with the recovered queue " + src.getPeerClusterZnode());
> this.oldsources.remove(src);
> this.zkHelper.deleteSource(src.getPeerClusterZnode(), false);
> {code}
> This works in the case where the recovered source is done and is calling this
> method, but when removing a peer it never calls terminate on thus it leaving
> it running.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4386) NPE in TaskMonitor
[ https://issues.apache.org/jira/browse/HBASE-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121687#comment-13121687 ] Hudson commented on HBASE-4386: --- Integrated in HBase-0.92 #47 (See [https://builds.apache.org/job/HBase-0.92/47/]) HBASE-4386 Fix a potential NPE in TaskMonitor todd : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/monitoring/TaskMonitor.java > NPE in TaskMonitor > -- > > Key: HBASE-4386 > URL: https://issues.apache.org/jira/browse/HBASE-4386 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Critical > Fix For: 0.92.0 > > Attachments: hbase-4386.txt > > > Saw the following hitting /rs-status > INTERNAL_SERVER_ERRORCaused > by:java.lang.NullPointerException > at > org.apache.hadoop.hbase.monitoring.TaskMonitor.purgeExpiredTasks(TaskMonitor.java:97) > at > org.apache.hadoop.hbase.monitoring.TaskMonitor.getTasks(TaskMonitor.java:127) > at > org.apache.hbase.tmpl.common.TaskMonitorTmplImpl.renderNoFlush(TaskMonitorTmplImpl.java:50) > at > org.apache.hbase.tmpl.common.TaskMonitorTmpl.renderNoFlush(TaskMonitorTmpl.java:170) > at > org.apache.hbase.tmpl.regionserver.RSStatusTmplImpl.renderNoFlush(RSStatusTmplImpl.java:70) > at > org.apache.hbase.tmpl.regionserver.RSStatusTmpl.renderNoFlush(RSStatusTmpl.java:176) > at > org.apache.hbase.tmpl.regionserver.RSStatusTmpl.render(RSStatusTmpl.java:167) > at > org.apache.hadoop.hbase.regionserver.RSStatusServlet.doGet(RSStatusServlet.java:48) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4402) Retaining locality after restart broken
[
https://issues.apache.org/jira/browse/HBASE-4402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122364#comment-13122364
]
Hudson commented on HBASE-4402:
---
Integrated in HBase-0.92 #48 (See
[https://builds.apache.org/job/HBase-0.92/48/])
HBASE-4402 Retaining locality after restart broken
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/DefaultLoadBalancer.java
*
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestDefaultLoadBalancer.java
> Retaining locality after restart broken
> ---
>
> Key: HBASE-4402
> URL: https://issues.apache.org/jira/browse/HBASE-4402
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.92.0
>Reporter: Todd Lipcon
>Assignee: Todd Lipcon
>Priority: Blocker
> Fix For: 0.92.0
>
> Attachments: 4402-v3.txt, hbase-4402.txt, hbase-4402.txt
>
>
> In DefaultLoadBalancer, we implement the "retain assignment" function like so:
> {code}
> if (sn != null && servers.contains(sn)) {
> assignments.get(sn).add(region.getKey());
> {code}
> but this will never work since after a cluster restart, all servers have a
> new ServerName with a new startcode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4482) Race Condition Concerning Eviction in SlabCache
[ https://issues.apache.org/jira/browse/HBASE-4482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122447#comment-13122447 ] Hudson commented on HBASE-4482: --- Integrated in HBase-0.92 #49 (See [https://builds.apache.org/job/HBase-0.92/49/]) HBASE-4482 Race Condition Concerning Eviction in SlabCache stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SingleSizeCache.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SlabCache.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SlabItemEvictionWatcher.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/CacheTestUtils.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/slab/TestSingleSizeCache.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/slab/TestSlabCache.java > Race Condition Concerning Eviction in SlabCache > --- > > Key: HBASE-4482 > URL: https://issues.apache.org/jira/browse/HBASE-4482 > Project: HBase > Issue Type: Sub-task >Reporter: Li Pi >Assignee: Li Pi >Priority: Blocker > Fix For: 0.92.0 > > Attachments: hbase-4482v1.txt, hbase-4482v2.txt, hbase-4482v4.2.txt, > hbase-4482v4.2.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4547) TestAdmin failing in 0.92 because .tableinfo not found
[
https://issues.apache.org/jira/browse/HBASE-4547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122446#comment-13122446
]
Hudson commented on HBASE-4547:
---
Integrated in HBase-0.92 #49 (See
[https://builds.apache.org/job/HBase-0.92/49/])
HBASE-4547 TestAdmin failing in 0.92 because .tableinfo not found
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
> TestAdmin failing in 0.92 because .tableinfo not found
> --
>
> Key: HBASE-4547
> URL: https://issues.apache.org/jira/browse/HBASE-4547
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 4547.txt
>
>
> I've been running tests before commit and found the following happens with
> some regularity, sporadic of course, but they fail fairly frequently:
> {code}
> Failed tests:
> testOnlineChangeTableSchema(org.apache.hadoop.hbase.client.TestAdmin)
> testForceSplit(org.apache.hadoop.hbase.client.TestAdmin): expected:<2> but
> was:<1>
> testForceSplitMultiFamily(org.apache.hadoop.hbase.client.TestAdmin):
> expected:<2> but was:<1>
> {code}
> Looking, it seems like we fail to find .tableinfo in the tests that modify
> table schema while table is online.
> The update of a table schema just does an overwrite. In the tests we
> sometimes fail to find the newly written file or we get EOFE reading it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4547) TestAdmin failing in 0.92 because .tableinfo not found
[
https://issues.apache.org/jira/browse/HBASE-4547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123175#comment-13123175
]
Hudson commented on HBASE-4547:
---
Integrated in HBase-0.92 #51 (See
[https://builds.apache.org/job/HBase-0.92/51/])
HBASE-4547 TestAdmin failing in 0.92 because .tableinfo not found
stack :
Files :
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
> TestAdmin failing in 0.92 because .tableinfo not found
> --
>
> Key: HBASE-4547
> URL: https://issues.apache.org/jira/browse/HBASE-4547
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 4547-part2.txt, 4547.txt
>
>
> I've been running tests before commit and found the following happens with
> some regularity, sporadic of course, but they fail fairly frequently:
> {code}
> Failed tests:
> testOnlineChangeTableSchema(org.apache.hadoop.hbase.client.TestAdmin)
> testForceSplit(org.apache.hadoop.hbase.client.TestAdmin): expected:<2> but
> was:<1>
> testForceSplitMultiFamily(org.apache.hadoop.hbase.client.TestAdmin):
> expected:<2> but was:<1>
> {code}
> Looking, it seems like we fail to find .tableinfo in the tests that modify
> table schema while table is online.
> The update of a table schema just does an overwrite. In the tests we
> sometimes fail to find the newly written file or we get EOFE reading it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4430) Disable TestSlabCache and TestSingleSizedCache temporarily to see if these are cause of build box failure though all tests pass
[ https://issues.apache.org/jira/browse/HBASE-4430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123623#comment-13123623 ] Hudson commented on HBASE-4430: --- Integrated in HBase-TRUNK #2309 (See [https://builds.apache.org/job/HBase-TRUNK/2309/]) HBASE-4430 Disable TestSlabCache and TestSingleSizedCache temporarily to see if these are cause of build box failure though all tests pass -- DISABLED TESTSLABCACHE AGAIN... its hanging stack : Files : * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/slab/TestSlabCache.java > Disable TestSlabCache and TestSingleSizedCache temporarily to see if these > are cause of build box failure though all tests pass > --- > > Key: HBASE-4430 > URL: https://issues.apache.org/jira/browse/HBASE-4430 > Project: HBase > Issue Type: Task > Components: test >Reporter: stack >Assignee: Li Pi > Fix For: 0.92.0 > > Attachments: TestSlabCache.trace > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4430) Disable TestSlabCache and TestSingleSizedCache temporarily to see if these are cause of build box failure though all tests pass
[ https://issues.apache.org/jira/browse/HBASE-4430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123643#comment-13123643 ] Hudson commented on HBASE-4430: --- Integrated in HBase-TRUNK #2310 (See [https://builds.apache.org/job/HBase-TRUNK/2310/]) HBASE-4430 Disable TestSlabCache and TestSingleSizedCache temporarily to see if these are cause of build box failure though all tests pass -- DISABLED TESTSLABCACHE AGAIN... its hanging stack : Files : * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/slab/TestSlabCache.java > Disable TestSlabCache and TestSingleSizedCache temporarily to see if these > are cause of build box failure though all tests pass > --- > > Key: HBASE-4430 > URL: https://issues.apache.org/jira/browse/HBASE-4430 > Project: HBase > Issue Type: Task > Components: test >Reporter: stack >Assignee: Li Pi > Fix For: 0.92.0 > > Attachments: TestSlabCache.trace > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4539) OpenedRegionHandler racing with itself in ServerShutDownhandler flow leading to HMaster abort
[ https://issues.apache.org/jira/browse/HBASE-4539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123657#comment-13123657 ] Hudson commented on HBASE-4539: --- Integrated in HBase-0.92 #54 (See [https://builds.apache.org/job/HBase-0.92/54/]) HBASE-4540 OpenedRegionHandler is not enforcing atomicity of the operation it is performing. Also addresses HBASE-4539. (Ram) ramkrishna : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestOpenedRegionHandler.java > OpenedRegionHandler racing with itself in ServerShutDownhandler flow leading > to HMaster abort > - > > Key: HBASE-4539 > URL: https://issues.apache.org/jira/browse/HBASE-4539 > Project: HBase > Issue Type: Bug >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > > Steps to reproduce > == > -> Region R1 is being opened in RS1. > ->After processing the znode to OPENED RS1 goes down. > ->Now before the OpenedRegionHandler executor deletes the znode if > ServerShutDownHandler tries to assign the region to RS2, RS2 transits the > node to OPENED and this OpenedRegionHandler executor deletes the znode. > ->Now if the first OpenedRegionHandler tries deleting the znode it throws > NoNode Exception and causes the HMaster to abort. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4540) OpenedRegionHandler is not enforcing atomicity of the operation it is performing
[
https://issues.apache.org/jira/browse/HBASE-4540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123658#comment-13123658
]
Hudson commented on HBASE-4540:
---
Integrated in HBase-0.92 #54 (See
[https://builds.apache.org/job/HBase-0.92/54/])
HBASE-4540 OpenedRegionHandler is not enforcing atomicity of the operation
it is performing. Also addresses HBASE-4539. (Ram)
ramkrishna :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
*
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestOpenedRegionHandler.java
> OpenedRegionHandler is not enforcing atomicity of the operation it is
> performing
>
>
> Key: HBASE-4540
> URL: https://issues.apache.org/jira/browse/HBASE-4540
> Project: HBase
> Issue Type: Bug
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-4540_1.patch
>
>
> -> OpenedRegionHandler has not yet deleted the znode of the region R1 opened
> by RS1.
> -> RS1 goes down.
> -> Servershutdownhandler assigns the region R1 to RS2.
> -> The znode of R1 is moved to OFFLINE state by master or OPENING state by
> RS2 if RS2 has started opening the region.
> -> Now the first OpenedRegionHandler tries to delete the znode thinking its
> in OPENED state but fails.
> -> Though it fails it removes the node from RIT and adds RS1 as the owner of
> R1 in master's memory.
> -> Now when RS2 completes opening the region the master is not able to open
> the region as already the reigon has been deleted from RIT.
> {code}
> Master
> ==
> 2011-10-05 20:49:45,301 INFO
> org.apache.hadoop.hbase.master.handler.ServerShutdownHandler: Finished
> processing of shutdown of linux146,60020,1317827727647
> 2011-10-05 20:49:54,177 DEBUG org.apache.hadoop.hbase.master.HMaster: Not
> running balancer because 1 region(s) in transition:
> {3e69d628a8bd8e9b7c5e7a2a6e03aad9=t1,,1317827883842.3e69d628a8bd8e9b7c5e7a2a6e03aad9.
> state=PENDING_OPEN, ts=1317827985272, server=linux76,60020,1317827746847}
> 2011-10-05 20:49:57,720 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=M_ZK_REGION_OFFLINE, server=linux76,6,1317827742012,
> region=3e69d628a8bd8e9b7c5e7a2a6e03aad9
> 2011-10-05 20:50:14,501 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> master:6-0x132d3dc13090023 Deleting existing unassigned node for
> 3e69d628a8bd8e9b7c5e7a2a6e03aad9 that is in expected state RS_ZK_REGION_OPENED
> 2011-10-05 20:50:14,505 WARN org.apache.hadoop.hbase.zookeeper.ZKAssign:
> master:6-0x132d3dc13090023 Attempting to delete unassigned node
> 3e69d628a8bd8e9b7c5e7a2a6e03aad9 in RS_ZK_REGION_OPENED state but node is in
> RS_ZK_REGION_OPENING state
> After the region is opened in RS2
> =
> 2011-10-05 20:50:48,066 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_OPENING, server=linux76,60020,1317827746847,
> region=3e69d628a8bd8e9b7c5e7a2a6e03aad9, which is more than 15 seconds late
> 2011-10-05 20:50:48,290 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received OPENING for region
> 3e69d628a8bd8e9b7c5e7a2a6e03aad9 from server linux76,60020,1317827746847 but
> region was in the state null and not in expected PENDING_OPEN or OPENING
> states
> 2011-10-05 20:50:53,743 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_OPENING, server=linux76,60020,1317827746847,
> region=3e69d628a8bd8e9b7c5e7a2a6e03aad9
> 2011-10-05 20:50:54,182 DEBUG org.apache.hadoop.hbase.master.CatalogJanitor:
> Scanned 1 catalog row(s) and gc'd 0 unreferenced parent region(s)
> 2011-10-05 20:50:54,397 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received OPENING for region
> 3e69d628a8bd8e9b7c5e7a2a6e03aad9 from server linux76,60020,1317827746847 but
> region was in the state null and not in expected PENDING_OPEN or OPENING
> states
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4539) OpenedRegionHandler racing with itself in ServerShutDownhandler flow leading to HMaster abort
[ https://issues.apache.org/jira/browse/HBASE-4539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123671#comment-13123671 ] Hudson commented on HBASE-4539: --- Integrated in HBase-TRUNK #2311 (See [https://builds.apache.org/job/HBase-TRUNK/2311/]) HBASE-4540 OpenedRegionHandler is not enforcing atomicity of the operation it is performing . Also fixes HBASE-4539 (ram) ramkrishna : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestOpenedRegionHandler.java > OpenedRegionHandler racing with itself in ServerShutDownhandler flow leading > to HMaster abort > - > > Key: HBASE-4539 > URL: https://issues.apache.org/jira/browse/HBASE-4539 > Project: HBase > Issue Type: Bug >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > > Steps to reproduce > == > -> Region R1 is being opened in RS1. > ->After processing the znode to OPENED RS1 goes down. > ->Now before the OpenedRegionHandler executor deletes the znode if > ServerShutDownHandler tries to assign the region to RS2, RS2 transits the > node to OPENED and this OpenedRegionHandler executor deletes the znode. > ->Now if the first OpenedRegionHandler tries deleting the znode it throws > NoNode Exception and causes the HMaster to abort. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4540) OpenedRegionHandler is not enforcing atomicity of the operation it is performing
[
https://issues.apache.org/jira/browse/HBASE-4540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123672#comment-13123672
]
Hudson commented on HBASE-4540:
---
Integrated in HBase-TRUNK #2311 (See
[https://builds.apache.org/job/HBase-TRUNK/2311/])
HBASE-4540 OpenedRegionHandler is not enforcing atomicity of the operation
it is performing . Also fixes HBASE-4539 (ram)
ramkrishna :
Files :
* /hbase/trunk/CHANGES.txt
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestOpenedRegionHandler.java
> OpenedRegionHandler is not enforcing atomicity of the operation it is
> performing
>
>
> Key: HBASE-4540
> URL: https://issues.apache.org/jira/browse/HBASE-4540
> Project: HBase
> Issue Type: Bug
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-4540_1.patch
>
>
> -> OpenedRegionHandler has not yet deleted the znode of the region R1 opened
> by RS1.
> -> RS1 goes down.
> -> Servershutdownhandler assigns the region R1 to RS2.
> -> The znode of R1 is moved to OFFLINE state by master or OPENING state by
> RS2 if RS2 has started opening the region.
> -> Now the first OpenedRegionHandler tries to delete the znode thinking its
> in OPENED state but fails.
> -> Though it fails it removes the node from RIT and adds RS1 as the owner of
> R1 in master's memory.
> -> Now when RS2 completes opening the region the master is not able to open
> the region as already the reigon has been deleted from RIT.
> {code}
> Master
> ==
> 2011-10-05 20:49:45,301 INFO
> org.apache.hadoop.hbase.master.handler.ServerShutdownHandler: Finished
> processing of shutdown of linux146,60020,1317827727647
> 2011-10-05 20:49:54,177 DEBUG org.apache.hadoop.hbase.master.HMaster: Not
> running balancer because 1 region(s) in transition:
> {3e69d628a8bd8e9b7c5e7a2a6e03aad9=t1,,1317827883842.3e69d628a8bd8e9b7c5e7a2a6e03aad9.
> state=PENDING_OPEN, ts=1317827985272, server=linux76,60020,1317827746847}
> 2011-10-05 20:49:57,720 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=M_ZK_REGION_OFFLINE, server=linux76,6,1317827742012,
> region=3e69d628a8bd8e9b7c5e7a2a6e03aad9
> 2011-10-05 20:50:14,501 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> master:6-0x132d3dc13090023 Deleting existing unassigned node for
> 3e69d628a8bd8e9b7c5e7a2a6e03aad9 that is in expected state RS_ZK_REGION_OPENED
> 2011-10-05 20:50:14,505 WARN org.apache.hadoop.hbase.zookeeper.ZKAssign:
> master:6-0x132d3dc13090023 Attempting to delete unassigned node
> 3e69d628a8bd8e9b7c5e7a2a6e03aad9 in RS_ZK_REGION_OPENED state but node is in
> RS_ZK_REGION_OPENING state
> After the region is opened in RS2
> =
> 2011-10-05 20:50:48,066 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_OPENING, server=linux76,60020,1317827746847,
> region=3e69d628a8bd8e9b7c5e7a2a6e03aad9, which is more than 15 seconds late
> 2011-10-05 20:50:48,290 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received OPENING for region
> 3e69d628a8bd8e9b7c5e7a2a6e03aad9 from server linux76,60020,1317827746847 but
> region was in the state null and not in expected PENDING_OPEN or OPENING
> states
> 2011-10-05 20:50:53,743 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_OPENING, server=linux76,60020,1317827746847,
> region=3e69d628a8bd8e9b7c5e7a2a6e03aad9
> 2011-10-05 20:50:54,182 DEBUG org.apache.hadoop.hbase.master.CatalogJanitor:
> Scanned 1 catalog row(s) and gc'd 0 unreferenced parent region(s)
> 2011-10-05 20:50:54,397 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received OPENING for region
> 3e69d628a8bd8e9b7c5e7a2a6e03aad9 from server linux76,60020,1317827746847 but
> region was in the state null and not in expected PENDING_OPEN or OPENING
> states
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4561) Update Maven documentation in book
[ https://issues.apache.org/jira/browse/HBASE-4561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123846#comment-13123846 ] Hudson commented on HBASE-4561: --- Integrated in HBase-TRUNK #2312 (See [https://builds.apache.org/job/HBase-TRUNK/2312/]) HBASE-4561 Update Maven documentation in book stack : Files : * /hbase/trunk/src/docbkx/developer.xml > Update Maven documentation in book > -- > > Key: HBASE-4561 > URL: https://issues.apache.org/jira/browse/HBASE-4561 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Jesse Yates >Priority: Minor > Attachments: book_HBASE-4561.txt > > > The maven documentation is a little out of date and has recently led to some > confusion about tests. This would cleanup the maven documents in the book to > be more explicit about how maven should be used. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4564) [book] book.xml,developer.xml - a few changes
[ https://issues.apache.org/jira/browse/HBASE-4564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13124158#comment-13124158 ] Hudson commented on HBASE-4564: --- Integrated in HBase-TRUNK #2314 (See [https://builds.apache.org/job/HBase-TRUNK/2314/]) HBASE-4564 book.xml,developer.xml dmeil : Files : * /hbase/trunk/src/docbkx/book.xml * /hbase/trunk/src/docbkx/developer.xml > [book] book.xml,developer.xml - a few changes > - > > Key: HBASE-4564 > URL: https://issues.apache.org/jira/browse/HBASE-4564 > Project: HBase > Issue Type: Improvement >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Minor > Attachments: docbkx_HBASE_4564.patch > > > book.xml > * Arch/Catalog - added comment that ROOT and META are actually HBase tables. > That came up this week on the dist-list. The book already describes the > structure but I'm making this even more explicit. > * Added comment on explicit versioning (setting timestamp) that this isn't > preferred (also came up on dist-list this week). > developer.xml > * adding entry for running multiple unit tests comma-listed (came up on > dist-list this week) > * adding link to ReviewBoard documentation in RB sub-section > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4566) [book] Documentation on KeyValue, section on capacity planning
[ https://issues.apache.org/jira/browse/HBASE-4566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13124448#comment-13124448 ] Hudson commented on HBASE-4566: --- Integrated in HBase-TRUNK #2315 (See [https://builds.apache.org/job/HBase-TRUNK/2315/]) HBASE-4566 book.xml,ops_mgt.xml - KeyValue documentation dmeil : Files : * /hbase/trunk/src/docbkx/book.xml * /hbase/trunk/src/docbkx/ops_mgt.xml > [book] Documentation on KeyValue, section on capacity planning > -- > > Key: HBASE-4566 > URL: https://issues.apache.org/jira/browse/HBASE-4566 > Project: HBase > Issue Type: Improvement >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Minor > Attachments: docbkx_HBASE_4566.patch > > > book.xml > * Added section in Arch/Regions on KeyValue and what it takes to store a KV. > * Added reference in Schema Design to new KV section. > ops_mgt.xml > * Created start of Capacity Planning section, emphasizing that KV storage > dominates the storage profile, with link to new KV section. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4571) [book] book.xml - added Schema Design Smackdown section
[ https://issues.apache.org/jira/browse/HBASE-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13124653#comment-13124653 ] Hudson commented on HBASE-4571: --- Integrated in HBase-TRUNK #2316 (See [https://builds.apache.org/job/HBase-TRUNK/2316/]) HBASE-4571. book.xml, adding Schema Design Smackdown section. dmeil : Files : * /hbase/trunk/src/docbkx/book.xml > [book] book.xml - added Schema Design Smackdown section > --- > > Key: HBASE-4571 > URL: https://issues.apache.org/jira/browse/HBASE-4571 > Project: HBase > Issue Type: Improvement >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Minor > Attachments: book_HBASE_4571.xml.patch > > > book.xml > * added Schema Design Smackdown section in the Schema Design chapter. Common > schema design questions on the dist-list. > * Rows vs. Versions > * Rows vs. Columns -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4573) [book] book.xml - adding examples in KeyValue section
[ https://issues.apache.org/jira/browse/HBASE-4573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13125296#comment-13125296 ] Hudson commented on HBASE-4573: --- Integrated in HBase-TRUNK #2317 (See [https://builds.apache.org/job/HBase-TRUNK/2317/]) HBASE-4573 book.xml, Put to KeyValue examples. dmeil : Files : * /hbase/trunk/src/docbkx/book.xml > [book] book.xml - adding examples in KeyValue section > - > > Key: HBASE-4573 > URL: https://issues.apache.org/jira/browse/HBASE-4573 > Project: HBase > Issue Type: Improvement >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Minor > Attachments: book_HBASE_4573.xml.patch > > > book.xml > * In Arch/Region/Store/KeyValue, adding a sub-section for examples of how a > Put is turned into a KeyValue. > * This is a follow-up from yesterday's patch where it added documentation on > KeyValue in the arch section. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4335) Splits can create temporary holes in .META. that confuse clients and regionservers
[ https://issues.apache.org/jira/browse/HBASE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13125492#comment-13125492 ] Hudson commented on HBASE-4335: --- Integrated in HBase-0.92 #55 (See [https://builds.apache.org/job/HBase-0.92/55/]) Integrating HBASE-4335 larsh : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransaction.java > Splits can create temporary holes in .META. that confuse clients and > regionservers > -- > > Key: HBASE-4335 > URL: https://issues.apache.org/jira/browse/HBASE-4335 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.90.4 >Reporter: Joe Pallas >Assignee: Lars Hofhansl >Priority: Critical > Fix For: 0.92.0 > > Attachments: 4335-v2.txt, 4335-v3.txt, 4335-v4.txt, 4335-v5.txt, > 4335.txt > > > When a SplitTransaction is performed, three updates are done to .META.: > 1. The parent region is marked as splitting (and hence offline) > 2. The first daughter region is added (same start key as parent) > 3. The second daughter region is added (split key is start key) > (later, the original parent region is deleted, but that's not important to > this discussion) > Steps 2 and 3 are actually done concurrently by > SplitTransaction.DaughterOpener threads. While the master is notified when a > split is complete, the only visibility that clients have is whether the > daughter regions have appeared in .META. > If the second daughter is added to .META. first, then .META. will contain the > (offline) parent region followed by the second daughter region. If the > client looks up a key that is greater than (or equal to) the split, the > client will find the second daughter region and use it. If the key is less > than the split key, the client will find the parent region and see that it is > offline, triggering a retry. > If the first daughter is added to .META. before the second daughter, there is > a window during which .META. has a hole: the first daughter effectively hides > the parent region (same start key), but there is no entry for the second > daughter. A region lookup will find the first daughter for all keys in the > parent's range, but the first daughter does not include keys at or beyond the > split key. > See HBASE-4333 and HBASE-4334 for details on how this causes problems and > suggestions for mitigating this in the client and regionserver. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4576) [book] moved Cluster Replication chapter to OpsMgt, other nit
[ https://issues.apache.org/jira/browse/HBASE-4576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13125540#comment-13125540 ] Hudson commented on HBASE-4576: --- Integrated in HBase-TRUNK #2318 (See [https://builds.apache.org/job/HBase-TRUNK/2318/]) HBASE-4576 book.xml,ops_mgt.xml - moved cluster repl into OpsMgt chapter. dmeil : Files : * /hbase/trunk/src/docbkx/book.xml * /hbase/trunk/src/docbkx/ops_mgt.xml > [book] moved Cluster Replication chapter to OpsMgt, other nit > - > > Key: HBASE-4576 > URL: https://issues.apache.org/jira/browse/HBASE-4576 > Project: HBase > Issue Type: Improvement >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Trivial > Attachments: docbkx_HBASE_4576.patch > > > book.xml > * There was a placeholder chapter for Cluster Replication that just linked to > replication.html. I moved this to a section in the Ops Mgt chapter. > * Nit: in Schema Design Smackdown sub-section, changed "Winner" to > "Preference". I got a little carried away when I created that entry and I'm > toning it down a tad. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4335) Splits can create temporary holes in .META. that confuse clients and regionservers
[ https://issues.apache.org/jira/browse/HBASE-4335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13125539#comment-13125539 ] Hudson commented on HBASE-4335: --- Integrated in HBase-TRUNK #2318 (See [https://builds.apache.org/job/HBase-TRUNK/2318/]) Integrating HBASE-4335 larsh : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransaction.java > Splits can create temporary holes in .META. that confuse clients and > regionservers > -- > > Key: HBASE-4335 > URL: https://issues.apache.org/jira/browse/HBASE-4335 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.90.4 >Reporter: Joe Pallas >Assignee: Lars Hofhansl >Priority: Critical > Fix For: 0.92.0 > > Attachments: 4335-v2.txt, 4335-v3.txt, 4335-v4.txt, 4335-v5.txt, > 4335.txt > > > When a SplitTransaction is performed, three updates are done to .META.: > 1. The parent region is marked as splitting (and hence offline) > 2. The first daughter region is added (same start key as parent) > 3. The second daughter region is added (split key is start key) > (later, the original parent region is deleted, but that's not important to > this discussion) > Steps 2 and 3 are actually done concurrently by > SplitTransaction.DaughterOpener threads. While the master is notified when a > split is complete, the only visibility that clients have is whether the > daughter regions have appeared in .META. > If the second daughter is added to .META. first, then .META. will contain the > (offline) parent region followed by the second daughter region. If the > client looks up a key that is greater than (or equal to) the split, the > client will find the second daughter region and use it. If the key is less > than the split key, the client will find the parent region and see that it is > offline, triggering a retry. > If the first daughter is added to .META. before the second daughter, there is > a window during which .META. has a hole: the first daughter effectively hides > the parent region (same start key), but there is no entry for the second > daughter. A region lookup will find the first daughter for all keys in the > parent's range, but the first daughter does not include keys at or beyond the > split key. > See HBASE-4333 and HBASE-4334 for details on how this causes problems and > suggestions for mitigating this in the client and regionserver. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4422) Move block cache parameters and references into single CacheConf class
[ https://issues.apache.org/jira/browse/HBASE-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13125614#comment-13125614 ] Hudson commented on HBASE-4422: --- Integrated in HBase-TRUNK #2319 (See [https://builds.apache.org/job/HBase-TRUNK/2319/]) HBASE-4422 Move block cache parameters and references into single CacheConf class (jgray) jgray : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCache.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/DoubleBlockCache.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/SimpleBlockCache.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SingleSizeCache.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SlabCache.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/BloomFilterFactory.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/CompressionTest.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/RandomSeek.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFilePerformance.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileReaderV1.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestSeekTo.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactSelection.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompoundBloomFilter.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFileBlockCacheSummary.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java > Move block cache parameters and references into single CacheConf class > -- > > Key: HBASE-4422 > URL: https://issues.apache.org/jira/browse/HBASE-4422 > Project: HBase > Issue Type: Improvement > Components: io >Reporter: Jonathan Gray >Assignee: Jonathan Gray > Fix For: 0.92.0 > > Attachments: CacheConfig92-v8.patch, HBASE-4422-FINAL-branch92.patch, > HBASE-4422-FINAL-trunk.patch > > > From StoreFile down to HFile, we currently u
[jira] [Commented] (HBASE-4422) Move block cache parameters and references into single CacheConf class
[ https://issues.apache.org/jira/browse/HBASE-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13125878#comment-13125878 ] Hudson commented on HBASE-4422: --- Integrated in HBase-0.92 #57 (See [https://builds.apache.org/job/HBase-0.92/57/]) HBASE-4422 Move block cache parameters and references into single CacheConf class (jgray) jgray : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCache.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/DoubleBlockCache.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/SimpleBlockCache.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SingleSizeCache.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SlabCache.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/BloomFilterFactory.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/CompressionTest.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/RandomSeek.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFilePerformance.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileReaderV1.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/io/hfile/TestSeekTo.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactSelection.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompoundBloomFilter.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFileBlockCacheSummary.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java > Move block cache parameters and references into single CacheConf class > -- > > Key: HBASE-4422 > URL: https://issues.apac
[jira] [Commented] (HBASE-4581) [book] changing revhistory, adding some ASF info to appendix
[ https://issues.apache.org/jira/browse/HBASE-4581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126056#comment-13126056 ] Hudson commented on HBASE-4581: --- Integrated in HBase-TRUNK #2320 (See [https://builds.apache.org/job/HBase-TRUNK/2320/]) HBASE-4581 book.xml (revhistory fix, asf-info), developer.xml dmeil : Files : * /hbase/trunk/src/docbkx/book.xml * /hbase/trunk/src/docbkx/developer.xml > [book] changing revhistory, adding some ASF info to appendix > > > Key: HBASE-4581 > URL: https://issues.apache.org/jira/browse/HBASE-4581 > Project: HBase > Issue Type: Improvement >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Minor > Attachments: docbkx_HBASE_4581.patch > > > book.xml > * refactored the revision history in the beginning of the book to be a single > entry that just says "HBase version" instead of the "initial checkin..." > stuff that was there before. > * added some info about the ASF in the appendix (with link to ASF development > page). > developer.xml > * added link in Getting Involved section to the appendix about ASF. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4582) Store.java cleanup (failing TestHeapSize and has warnings)
[ https://issues.apache.org/jira/browse/HBASE-4582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126151#comment-13126151 ] Hudson commented on HBASE-4582: --- Integrated in HBase-TRUNK #2321 (See [https://builds.apache.org/job/HBase-TRUNK/2321/]) HBASE-4582 Store.java cleanup (failing TestHeapSize and has warnings) (jgray) jgray : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java > Store.java cleanup (failing TestHeapSize and has warnings) > -- > > Key: HBASE-4582 > URL: https://issues.apache.org/jira/browse/HBASE-4582 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.92.0, 0.94.0 >Reporter: Jonathan Gray >Assignee: Jonathan Gray >Priority: Minor > Fix For: 0.92.0 > > Attachments: HBASE-4582-FINAL.patch, HBASE-4582-v1-branch92.patch > > > I broke TestHeapSize on 32-bit when I committed HBASE-4422. Fix it here, > remove another boolean variable I forgot to delete, and fix a couple warnings > while we are at it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4555) TestShell seems passed, but actually errors seen in test output file
[
https://issues.apache.org/jira/browse/HBASE-4555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126150#comment-13126150
]
Hudson commented on HBASE-4555:
---
Integrated in HBase-TRUNK #2321 (See
[https://builds.apache.org/job/HBase-TRUNK/2321/])
HBASE-4555 TestShell seems passed, but actually errors seen in test output
file
stack :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/test/ruby/hbase/admin_test.rb
* /hbase/trunk/src/test/ruby/tests_runner.rb
> TestShell seems passed, but actually errors seen in test output file
>
>
> Key: HBASE-4555
> URL: https://issues.apache.org/jira/browse/HBASE-4555
> Project: HBase
> Issue Type: Test
> Components: test
>Reporter: Mingjie Lai
>Assignee: Mingjie Lai
> Fix For: 0.92.0
>
> Attachments: 4555.diff, HBase-4555-trunk-0.92.patch
>
>
> When I was making test cases for 4554, I saw a weird issue that TestShell
> seems to pass, but actually I saw error messages in the output file.
> {code}
> ---
> T E S T S
> ---
> Running org.apache.hadoop.hbase.client.TestShell
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.252 sec
> Results :
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
> {code}
> Error messages in org.apache.hadoop.hbase.client.TestShell-output.txt:
> {code}
> ...
> 6) Error:
> test_alter_should_support_shortcut_DELETE_alter_specs(Hbase::AdminAlterTableTest):
> ArgumentError: There should be at least one argument but the table name
>
> /home/mlai/git/hbase-private/src/test/ruby/../../main/ruby/hbase/admin.rb:307:in
> `alter'
> ./src/test/ruby/hbase/admin_test.rb:271:in
> `test_alter_should_support_shortcut_DELETE_alter_specs'
> org/jruby/RubyProc.java:268:in `call'
> org/jruby/RubyKernel.java:2038:in `send'
> org/jruby/RubyArray.java:1572:in `each'
> org/jruby/RubyArray.java:1572:in `each'
>
> 7) Error:
> test_split_should_work(Hbase::AdminMethodsTest):
> ArgumentError: wrong number of arguments (1 for 2)
> ./src/test/ruby/hbase/admin_test.rb:99:in `test_split_should_work'
> org/jruby/RubyProc.java:268:in `call'
> org/jruby/RubyKernel.java:2038:in `send'
> org/jruby/RubyArray.java:1572:in `each'
> org/jruby/RubyArray.java:1572:in `each'
>
> 192 tests, 259 assertions, 1 failures, 6 errors
> Done with tests! Shutting down the cluster...
> 2011-10-07 16:46:14,760 INFO [main] hbase.HBaseTestingUtility(551): Shutting
> down minicluster
> 2011-10-07 16:46:14,760 DEBUG [main] util.JVMClusterUtil(214): Shutting down
> HBase Cluster
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4582) Store.java cleanup (failing TestHeapSize and has warnings)
[ https://issues.apache.org/jira/browse/HBASE-4582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126237#comment-13126237 ] Hudson commented on HBASE-4582: --- Integrated in HBase-0.92 #58 (See [https://builds.apache.org/job/HBase-0.92/58/]) HBASE-4582 Store.java cleanup (failing TestHeapSize and has warnings) (jgray) jgray : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java > Store.java cleanup (failing TestHeapSize and has warnings) > -- > > Key: HBASE-4582 > URL: https://issues.apache.org/jira/browse/HBASE-4582 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.92.0, 0.94.0 >Reporter: Jonathan Gray >Assignee: Jonathan Gray >Priority: Minor > Fix For: 0.92.0 > > Attachments: HBASE-4582-FINAL.patch, HBASE-4582-v1-branch92.patch > > > I broke TestHeapSize on 32-bit when I committed HBASE-4422. Fix it here, > remove another boolean variable I forgot to delete, and fix a couple warnings > while we are at it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4555) TestShell seems passed, but actually errors seen in test output file
[
https://issues.apache.org/jira/browse/HBASE-4555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126236#comment-13126236
]
Hudson commented on HBASE-4555:
---
Integrated in HBase-0.92 #58 (See
[https://builds.apache.org/job/HBase-0.92/58/])
HBASE-4555 TestShell seems passed, but actually errors seen in test output
file
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/test/ruby/hbase/admin_test.rb
* /hbase/branches/0.92/src/test/ruby/tests_runner.rb
> TestShell seems passed, but actually errors seen in test output file
>
>
> Key: HBASE-4555
> URL: https://issues.apache.org/jira/browse/HBASE-4555
> Project: HBase
> Issue Type: Test
> Components: test
>Reporter: Mingjie Lai
>Assignee: Mingjie Lai
> Fix For: 0.92.0
>
> Attachments: 4555.diff, HBase-4555-trunk-0.92.patch
>
>
> When I was making test cases for 4554, I saw a weird issue that TestShell
> seems to pass, but actually I saw error messages in the output file.
> {code}
> ---
> T E S T S
> ---
> Running org.apache.hadoop.hbase.client.TestShell
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 39.252 sec
> Results :
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
> {code}
> Error messages in org.apache.hadoop.hbase.client.TestShell-output.txt:
> {code}
> ...
> 6) Error:
> test_alter_should_support_shortcut_DELETE_alter_specs(Hbase::AdminAlterTableTest):
> ArgumentError: There should be at least one argument but the table name
>
> /home/mlai/git/hbase-private/src/test/ruby/../../main/ruby/hbase/admin.rb:307:in
> `alter'
> ./src/test/ruby/hbase/admin_test.rb:271:in
> `test_alter_should_support_shortcut_DELETE_alter_specs'
> org/jruby/RubyProc.java:268:in `call'
> org/jruby/RubyKernel.java:2038:in `send'
> org/jruby/RubyArray.java:1572:in `each'
> org/jruby/RubyArray.java:1572:in `each'
>
> 7) Error:
> test_split_should_work(Hbase::AdminMethodsTest):
> ArgumentError: wrong number of arguments (1 for 2)
> ./src/test/ruby/hbase/admin_test.rb:99:in `test_split_should_work'
> org/jruby/RubyProc.java:268:in `call'
> org/jruby/RubyKernel.java:2038:in `send'
> org/jruby/RubyArray.java:1572:in `each'
> org/jruby/RubyArray.java:1572:in `each'
>
> 192 tests, 259 assertions, 1 failures, 6 errors
> Done with tests! Shutting down the cluster...
> 2011-10-07 16:46:14,760 INFO [main] hbase.HBaseTestingUtility(551): Shutting
> down minicluster
> 2011-10-07 16:46:14,760 DEBUG [main] util.JVMClusterUtil(214): Shutting down
> HBase Cluster
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4102) atomicAppend: A put that appends to the latest version of a cell; i.e. reads current value then adds the bytes offered by the client to the tail and writes out a new en
[ https://issues.apache.org/jira/browse/HBASE-4102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126274#comment-13126274 ] Hudson commented on HBASE-4102: --- Integrated in HBase-TRUNK #2322 (See [https://builds.apache.org/job/HBase-TRUNK/2322/]) HBASE-4102 atomicAppend: A put that appends to the latest version of a cell larsh : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/KeyValue.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Append.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Mutation.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Put.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestIncrement.java > atomicAppend: A put that appends to the latest version of a cell; i.e. reads > current value then adds the bytes offered by the client to the tail and > writes out a new entry > --- > > Key: HBASE-4102 > URL: https://issues.apache.org/jira/browse/HBASE-4102 > Project: HBase > Issue Type: New Feature >Reporter: stack >Assignee: Lars Hofhansl > Fix For: 0.94.0 > > Attachments: 4102-v1.txt, 4102.txt > > > Its come up a few times that clients want to add to an existing cell rather > than make a new cell each time. At our place, the frontend keeps a list of > urls a user has visited -- their md5s -- and updates it as user progresses. > Rather than read, modify client-side, then write new value back to hbase, it > would be sweet if could do it all in one operation in hbase server. TSDB > aims to be space efficient. Rather than pay the cost of the KV wrapper per > metric, it would rather have a KV for an interval an in this KV have a value > that is all the metrics for the period. > It could be done as a coprocessor but this feels more like a fundamental > feature. > Benoît suggests that atomicAppend take a flag to indicate whether or not the > client wants to see the resulting cell; often a client won't want to see the > result and in this case, why pay the price formulating and delivering a > response that client just drops. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4584) [book] moved Metrics chapter under OpsMgt, moved Building HBase under Developing HBase
[ https://issues.apache.org/jira/browse/HBASE-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126309#comment-13126309 ] Hudson commented on HBASE-4584: --- Integrated in HBase-TRUNK #2323 (See [https://builds.apache.org/job/HBase-TRUNK/2323/]) HBASE-4584. some chapter reorganization (book.xml, developer.xml, ops_mgt.xml). dmeil : Files : * /hbase/trunk/src/docbkx/book.xml * /hbase/trunk/src/docbkx/build.xml * /hbase/trunk/src/docbkx/developer.xml * /hbase/trunk/src/docbkx/ops_mgt.xml > [book] moved Metrics chapter under OpsMgt, moved Building HBase under > Developing HBase > -- > > Key: HBASE-4584 > URL: https://issues.apache.org/jira/browse/HBASE-4584 > Project: HBase > Issue Type: Improvement >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Minor > Attachments: docbkx_HBASE_4584.patch > > > book changes. chapter re-organization. > book.xml, ops_mgt.xml > * moved metrics chapter from book to a section in OpsMgt chapter > build.xml, developer.xml > * moved the small "Building HBase" chapter into "Developing HBase" chapter as > a section. > * deleted build.xml > * renamed "Developing HBase" chapter to "Building and Developing HBase" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4558) Refactor TestOpenedRegionHandler and TestOpenRegionHandler.
[ https://issues.apache.org/jira/browse/HBASE-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126461#comment-13126461 ] Hudson commented on HBASE-4558: --- Integrated in HBase-0.92 #60 (See [https://builds.apache.org/job/HBase-0.92/60/]) HBASE-4558 - Addendum for TestMasterFailover (Ram) HBASE-4558 Refactor TestOpenedRegionHandler and TestOpenRegionHandler. (Ram) ramkrishna : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java ramkrishna : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestOpenedRegionHandler.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/handler/MockRegionServerServices.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/handler/MockServer.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestCloseRegionHandler.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/MockServer.java > Refactor TestOpenedRegionHandler and TestOpenRegionHandler. > --- > > Key: HBASE-4558 > URL: https://issues.apache.org/jira/browse/HBASE-4558 > Project: HBase > Issue Type: Improvement >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 0.92.0 > > Attachments: HBASE-4558_1.patch, HBASE-4558_2.patch, > HBASE-4558_3.patch > > > This is an improvement task taken up to refactor TestOpenedRegionandler and > TestOpenRegionHandler so that MockServer and MockRegionServerServices can be > accessed from a common utility package. > If we do this then one of the testcases in TestOpenedRegionHandler need not > start up a cluster and also moving it into a common package will help in > mocking the server for future testcases. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4556) Fix all incorrect uses of InternalScanner.next(...)
[
https://issues.apache.org/jira/browse/HBASE-4556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126873#comment-13126873
]
Hudson commented on HBASE-4556:
---
Integrated in HBase-0.92 #61 (See
[https://builds.apache.org/job/HBase-0.92/61/])
HBASE-4556 Fix all incorrect uses of InternalScanner.next(...)
larsh :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/HMerge.java
> Fix all incorrect uses of InternalScanner.next(...)
> ---
>
> Key: HBASE-4556
> URL: https://issues.apache.org/jira/browse/HBASE-4556
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Attachments: 4556-v1.txt, 4556.txt
>
>
> There are cases all over the code where InternalScanner.next(...) is not used
> correctly.
> I see this a lot:
> {code}
> while(scanner.next(...)) {
> }
> {code}
> The correct pattern is:
> {code}
> boolean more = false;
> do {
>more = scanner.next(...);
> } while (more);
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4078) Silent Data Offlining During HDFS Flakiness
[ https://issues.apache.org/jira/browse/HBASE-4078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127103#comment-13127103 ] Hudson commented on HBASE-4078: --- Integrated in HBase-0.92 #62 (See [https://builds.apache.org/job/HBase-0.92/62/]) HBASE-4078 Validate store files after flush/compaction nspiegelberg : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java > Silent Data Offlining During HDFS Flakiness > --- > > Key: HBASE-4078 > URL: https://issues.apache.org/jira/browse/HBASE-4078 > Project: HBase > Issue Type: Bug > Components: io, regionserver >Affects Versions: 0.89.20100924, 0.90.3, 0.92.0 >Reporter: Nicolas Spiegelberg >Assignee: Pritam Damania >Priority: Blocker > Fix For: 0.92.0, 0.94.0 > > Attachments: > 0001-Validate-store-files-after-compactions-flushes.patch, > 0001-Validate-store-files.patch > > > See HBASE-1436 . The bug fix for this JIRA is a temporary workaround for > improperly moving partially-written files from TMP into the region directory > when a FS error occurs. Unfortunately, the fix is to ignore all IO > exceptions, which masks off-lining due to FS flakiness. We need to > permanently fix the problem that created HBASE-1436 & then at least have the > option to not open a region during times of flakey FS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4551) Small fixes to compile against 0.23-SNAPSHOT
[ https://issues.apache.org/jira/browse/HBASE-4551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127196#comment-13127196 ] Hudson commented on HBASE-4551: --- Integrated in HBase-0.92 #63 (See [https://builds.apache.org/job/HBase-0.92/63/]) HBASE-4551 Fix pom and some test cases to compile and run against Hadoop 0.23 todd : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/pom.xml * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/coprocessor/TestWALObserver.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java > Small fixes to compile against 0.23-SNAPSHOT > > > Key: HBASE-4551 > URL: https://issues.apache.org/jira/browse/HBASE-4551 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 0.92.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon > Fix For: 0.92.0 > > Attachments: hbase-4551.txt, hbase-4551.txt > > > - fix pom.xml to properly pull the test artifacts > - fix TestHLog to not use the private cluster.getNameNode() API -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3417) CacheOnWrite is using the temporary output path for block names, need to use a more consistent block naming scheme
[ https://issues.apache.org/jira/browse/HBASE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127197#comment-13127197 ] Hudson commented on HBASE-3417: --- Integrated in HBase-0.92 #63 (See [https://builds.apache.org/job/HBase-0.92/63/]) HBASE-3417 CacheOnWrite is using the temporary output path for block names, need to use a more consistent block naming scheme (jgray) jgray : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java > CacheOnWrite is using the temporary output path for block names, need to use > a more consistent block naming scheme > -- > > Key: HBASE-3417 > URL: https://issues.apache.org/jira/browse/HBASE-3417 > Project: HBase > Issue Type: Bug > Components: io, regionserver >Affects Versions: 0.92.0 >Reporter: Jonathan Gray >Assignee: Jonathan Gray >Priority: Critical > Fix For: 0.92.0 > > Attachments: HBASE-3417-redux-v1.patch, HBASE-3417-v1.patch, > HBASE-3417-v2.patch, HBASE-3417-v5.patch > > > Currently the block names used in the block cache are built using the > filesystem path. However, for cache on write, the path is a temporary output > file. > The original COW patch actually made some modifications to block naming stuff > to make it more consistent but did not do enough. Should add a separate > method somewhere for generating block names using some more easily mocked > scheme (rather than just raw path as we generate a random unique file name > twice, once for tmp and then again when moved into place). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4568) Make zk dump jsp response more quickly
[ https://issues.apache.org/jira/browse/HBASE-4568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127252#comment-13127252 ] Hudson commented on HBASE-4568: --- Integrated in HBase-0.92 #64 (See [https://builds.apache.org/job/HBase-0.92/64/]) HBASE-4568 Make zk dump jsp response faster nspiegelberg : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/RetryCounter.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java * /hbase/branches/0.92/src/main/resources/hbase-webapps/master/zk.jsp > Make zk dump jsp response more quickly > -- > > Key: HBASE-4568 > URL: https://issues.apache.org/jira/browse/HBASE-4568 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Liyin Tang > Fix For: 0.92.0, 0.94.0 > > Attachments: HBASE-4568.patch > > > 1) For each zk dump, currently hbase will create a zk client instance every > time. > This is quite slow when any machines in the quorum is dead. Because it will > connect to each machine in the zk quorum again. > > HMaster master = (HMaster)getServletContext().getAttribute(HMaster.MASTER); > Configuration conf = master.getConfiguration(); > HBaseAdmin hbadmin = new HBaseAdmin(conf); > HConnection connection = hbadmin.getConnection(); > ZooKeeperWatcher watcher = connection.getZooKeeperWatcher(); > > So we can simplify this: > > HMaster master = (HMaster)getServletContext().getAttribute(HMaster.MASTER); > ZooKeeperWatcher watcher = master.getZooKeeperWatcher(); > > 2) Also when hbase call getServerStats() for each machine in the zk quorum, > it hard coded the default time out as 1 min. > It would be nice to make this configurable and set it to a low time out. > When hbase tries to connect to each machine in the zk quorum, it will create > the socket, and then set the socket time out, and read it with this time out. > It means hbase will create a socket and connect to the zk server with 0 time > out at first, which will take a long time. > Because a timeout of zero is interpreted as an infinite timeout. The > connection will then block until established or an error occurs. > 3) The recoverable zookeeper should be real exponentially backoff when there > is connection loss exception, which will give hbase much longer time window > to recover from zk machine failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3446) ProcessServerShutdown fails if META moves, orphaning lots of regions
[ https://issues.apache.org/jira/browse/HBASE-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127253#comment-13127253 ] Hudson commented on HBASE-3446: --- Integrated in HBase-0.92 #64 (See [https://builds.apache.org/job/HBase-0.92/64/]) HBASE-3446 ProcessServerShutdown fails if META moves, orphaning lots of regions HBASE-3446 ProcessServerShutdown fails if META moves, orphaning lots of regions stack : Files : * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/catalog/MetaMigrationRemovingHTD.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/KeyValue.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/HTable.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/Result.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedException.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/handler/EnableTableHandler.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServer.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/MetaNodeTracker.java * /hbase/branches/0.92/src/main/ruby/hbase/admin.rb * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestMetaMigration.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java * /hbase/branches/0.92/src/test/ruby/hbase/admin_test.rb * /hbase/branches/0.92/src/test/ruby/shell/shell_test.rb > ProcessServerShutdown fails if META moves, orphaning lots of regions > > > Key: HBASE-3446 > URL: https://issues.apache.org/jira/browse/HBASE-3446 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.0 >Reporter: Todd Lipcon >Assignee: stack >Priority: Blocker > Fix For: 0.92.0 > > Attachments: 3446-v11.txt, 3446-v12.txt, 3446-v13.txt, 3446-v14.txt, > 3446-v2.txt, 3446-v3.txt, 3446-v4.txt, 3446-v7.txt, 3446-v9.txt, 3446.txt, > 3446v15.txt, 3446v23.txt > > > I ran a rolling restart on a 5 node cluster with lots of regions, and > afterwards had LOTS of regions left orphaned. The issue appears to b
[jira] [Commented] (HBASE-4282) RegionServer should abort when WAL close encounters an error with unflushed edits
[ https://issues.apache.org/jira/browse/HBASE-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127254#comment-13127254 ] Hudson commented on HBASE-4282: --- Integrated in HBase-0.92 #64 (See [https://builds.apache.org/job/HBase-0.92/64/]) HBASE-4282 RegionServer should abort when WAL close fails with unflushed edits garyh : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRollAbort.java > RegionServer should abort when WAL close encounters an error with unflushed > edits > - > > Key: HBASE-4282 > URL: https://issues.apache.org/jira/browse/HBASE-4282 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.0, 0.94.0, 0.90.5 >Reporter: Gary Helmling >Assignee: Gary Helmling >Priority: Blocker > Fix For: 0.92.0, 0.94.0, 0.90.5 > > Attachments: HBASE-4282_0.90_2.patch, HBASE-4282_0.90_final.patch, > HBASE-4282_0.92_final.patch, HBASE-4282_trunk_2.patch, > HBASE-4282_trunk_3.patch, HBASE-4282_trunk_final.patch, > HBASE-4282_trunk_prelim.patch > > > The ability to ride over WAL close errors on log rolling added in HBASE-4222 > could lead to missing HLog entries if: > * A table has DEFERRED_LOG_FLUSH=true > * There are unflushed WALEdit entries for that table in the current > SequenceFile writer buffer > Since the writes were already acknowledged to the client, just ignoring the > close error to allow for another log roll doesn't seem like the right thing > to do here. > We could easily flag this state and only ride over the close error if there > aren't unflushed entries. This would bring the above condition back to the > previous behavior of aborting the region server. However, aborting the > region server in this state is still guaranteeing data loss. Is there > anything we can do better in this case? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4589) CacheOnWrite broken in some cases because it can conflict with evictOnClose
[ https://issues.apache.org/jira/browse/HBASE-4589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128015#comment-13128015 ] Hudson commented on HBASE-4589: --- Integrated in HBase-0.92 #65 (See [https://builds.apache.org/job/HBase-0.92/65/]) HBASE-4589 CacheOnWrite broken in some cases because it can conflict with evictOnClose (jgray) jgray : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompoundBloomFilter.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java > CacheOnWrite broken in some cases because it can conflict with evictOnClose > --- > > Key: HBASE-4589 > URL: https://issues.apache.org/jira/browse/HBASE-4589 > Project: HBase > Issue Type: Bug > Components: io >Affects Versions: 0.92.0, 0.94.0 >Reporter: Jonathan Gray >Assignee: Jonathan Gray >Priority: Critical > Fix For: 0.92.0 > > Attachments: HBASE-4589-v1.patch > > > Commit of HBASE-4078 added some extra StoreFile verification which just did > an open of a StoreFile reader and then closes it, ensuring there's no > exception. If evict-on-close is on, which it is by default, this causes all > blocks of a file to be evicted even though it's still open. > We need to add the boolean into the close call in the way we have booleans > for cacheBlocks at some point since we need to make localized decisions in > some cases. > In lots of places, we can always rely on cacheConf.shouldEvictOnClose() so > shouldn't be too burdensome. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
