[jira] [Assigned] (HBASE-13798) TestFromClientSide* don't close the Table
[ https://issues.apache.org/jira/browse/HBASE-13798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi reassigned HBASE-13798: --- Assignee: Dima Spivak TestFromClientSide* don't close the Table - Key: HBASE-13798 URL: https://issues.apache.org/jira/browse/HBASE-13798 Project: HBase Issue Type: Bug Components: test Affects Versions: 2.0.0, 1.1.0 Reporter: Matteo Bertozzi Assignee: Dima Spivak Priority: Trivial TestFromClientSide, TestFromClientSide3 have lots of tests with the Table object not closed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13476) Procedure v2 - Add Replay Order logic for child procedures
[ https://issues.apache.org/jira/browse/HBASE-13476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563813#comment-14563813 ] Hudson commented on HBASE-13476: SUCCESS: Integrated in HBase-1.2 #111 (See [https://builds.apache.org/job/HBase-1.2/111/]) HBASE-13476 Procedure v2 - Add Replay Order logic for child procedures (matteo.bertozzi: rev 24ef755f83faf17ff35735badac66f0c8d250a5a) * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/ProcedureWALFormat.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/ProcedureStore.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/wal/TestWALProcedureStore.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/ProcedureTestingUtility.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureRecovery.java * hbase-server/src/test/resources/hbase-site.xml * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureExecution.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/ProcedureWALFormatReader.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureConstants.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureReplayOrder.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java Procedure v2 - Add Replay Order logic for child procedures -- Key: HBASE-13476 URL: https://issues.apache.org/jira/browse/HBASE-13476 Project: HBase Issue Type: Sub-task Components: proc-v2 Affects Versions: 2.0.0, 1.1.0 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Fix For: 2.0.0, 1.2.0 Attachments: HBASE-13476-v0.patch, HBASE-13476-v1.patch, HBASE-13476-v2.patch The current replay order logic is only for single-level procedures (which is what we are using today for master operations). To complete the implementation for the notification-bus we need to be able to replay in correct order child procs too. this will not impact the the current procs implementation (create/delete/modify/...) it is just a change at the framework level. https://reviews.apache.org/r/34289/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13797) Fix resource leak in HBaseFsck
[
https://issues.apache.org/jira/browse/HBASE-13797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563815#comment-14563815
]
Stephen Yuan Jiang commented on HBASE-13797:
+1 LGTM
Fix resource leak in HBaseFsck
--
Key: HBASE-13797
URL: https://issues.apache.org/jira/browse/HBASE-13797
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Attachments: 13797-v1.txt
In HBaseFsck#unassignMetaReplica() , ZooKeeperWatcher is not closed upon
return from the method:
{code}
ZKUtil.deleteNode(zkw,
zkw.getZNodeForReplica(hi.metaEntry.getReplicaId()));
{code}
This leads to resource leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13770) Programmatic JAAS configuration option for secure zookeeper may be broken
[ https://issues.apache.org/jira/browse/HBASE-13770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563840#comment-14563840 ] Gabor Liptak commented on HBASE-13770: -- Would https://github.com/apache/hbase/blob/master/hbase-server/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZooKeeperACL.java contain the right code? Programmatic JAAS configuration option for secure zookeeper may be broken - Key: HBASE-13770 URL: https://issues.apache.org/jira/browse/HBASE-13770 Project: HBase Issue Type: Bug Affects Versions: 2.0.0, 1.0.1, 1.1.0, 0.98.13, 1.2.0 Reporter: Andrew Purtell While verifying the patch fix for HBASE-13768 we were unable to successfully test the programmatic JAAS configuration option for secure ZooKeeper integration. Unclear if that was due to a bug or incorrect test configuration. Update the security section of the online book with clear instructions for setting up the programmatic JAAS configuration option for secure ZooKeeper integration. Verify it works. Fix as necessary. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13761) Optimize FuzzyRowFilter
[ https://issues.apache.org/jira/browse/HBASE-13761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563848#comment-14563848 ] Hudson commented on HBASE-13761: SUCCESS: Integrated in HBase-0.98 #1008 (See [https://builds.apache.org/job/HBase-0.98/1008/]) HBASE-13761 Optimize FuzzyRowFilter (Vladimir Rodionov) (ndimiduk: rev 04ede1cd98858f1fe8ace89e88320b0c5ef0f5db) * dev-support/test-patch.properties * hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFuzzyRowFilterEndToEnd.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FuzzyRowFilter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFuzzyRowFilter.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/UnsafeAccess.java Optimize FuzzyRowFilter --- Key: HBASE-13761 URL: https://issues.apache.org/jira/browse/HBASE-13761 Project: HBase Issue Type: Improvement Components: Filters Affects Versions: 2.0.0, 1.1.0, 0.98.13 Reporter: Vladimir Rodionov Assignee: Vladimir Rodionov Priority: Minor Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.1 Attachments: 13761-v8.patch, HBASE-13761-0.98.patch, HBASE-13761-branch-1.patch, HBASE-13761.patch, HBASE-13761_2.patch, HBASE-13761_4.patch, HBASE-13761_5.patch, HBASE-13761_6.patch, HBASE-13761_7.patch FuzzyRowFilter has some room for improvements: a lot of byte-by-byte arithmetic, non-efficient algorithm of selecting next candidate row etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13797) Fix resource leak in HBaseFsck
[
https://issues.apache.org/jira/browse/HBASE-13797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-13797:
---
Fix Version/s: 1.1.1
1.2.0
2.0.0
Fix resource leak in HBaseFsck
--
Key: HBASE-13797
URL: https://issues.apache.org/jira/browse/HBASE-13797
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Fix For: 2.0.0, 1.2.0, 1.1.1
Attachments: 13797-v1.txt
In HBaseFsck#unassignMetaReplica() , ZooKeeperWatcher is not closed upon
return from the method:
{code}
ZKUtil.deleteNode(zkw,
zkw.getZNodeForReplica(hi.metaEntry.getReplicaId()));
{code}
This leads to resource leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13761) Optimize FuzzyRowFilter
[ https://issues.apache.org/jira/browse/HBASE-13761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563864#comment-14563864 ] Hudson commented on HBASE-13761: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #959 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/959/]) HBASE-13761 Optimize FuzzyRowFilter (Vladimir Rodionov) (ndimiduk: rev 04ede1cd98858f1fe8ace89e88320b0c5ef0f5db) * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FuzzyRowFilter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFuzzyRowFilterEndToEnd.java * hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFuzzyRowFilter.java * dev-support/test-patch.properties * hbase-common/src/main/java/org/apache/hadoop/hbase/util/UnsafeAccess.java Optimize FuzzyRowFilter --- Key: HBASE-13761 URL: https://issues.apache.org/jira/browse/HBASE-13761 Project: HBase Issue Type: Improvement Components: Filters Affects Versions: 2.0.0, 1.1.0, 0.98.13 Reporter: Vladimir Rodionov Assignee: Vladimir Rodionov Priority: Minor Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.1 Attachments: 13761-v8.patch, HBASE-13761-0.98.patch, HBASE-13761-branch-1.patch, HBASE-13761.patch, HBASE-13761_2.patch, HBASE-13761_4.patch, HBASE-13761_5.patch, HBASE-13761_6.patch, HBASE-13761_7.patch FuzzyRowFilter has some room for improvements: a lot of byte-by-byte arithmetic, non-efficient algorithm of selecting next candidate row etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13787) use a cli argument parsing library instead of ad-hoc string handling
[ https://issues.apache.org/jira/browse/HBASE-13787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563866#comment-14563866 ] Gabor Liptak commented on HBASE-13787: -- https://github.com/apache/hbase/blob/master/pom.xml pulls in http://commons.apache.org/proper/commons-cli/javadocs/api-1.2/ use a cli argument parsing library instead of ad-hoc string handling Key: HBASE-13787 URL: https://issues.apache.org/jira/browse/HBASE-13787 Project: HBase Issue Type: Sub-task Components: mapreduce, util Reporter: Sean Busbey several of our mapreduce based tools manually parse strings or rely on system properties for hteir configuration. update all to use the same cli argument parsing library. The particular library used isn't important, but it should be one we already bring in for some other reason. If possible try to make the arguments consistent across all the tools, since you'll be looking broadly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13799) javadoc how Scan gets polluted when used; if you set attributes or ask for scan metrics
stack created HBASE-13799: - Summary: javadoc how Scan gets polluted when used; if you set attributes or ask for scan metrics Key: HBASE-13799 URL: https://issues.apache.org/jira/browse/HBASE-13799 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Matteo is noticing that Get and Scan get altered when used. Doc it for Scans (Matteo is 'fixing' it for Get). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13796) ZKUtil doesn't clean quorum setting properly
[ https://issues.apache.org/jira/browse/HBASE-13796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Geoffrey Jacoby updated HBASE-13796: Attachment: HBASE-13796.patch Patch to resolve the issue, plus added a covering test since the existing code seemed to lack one. ZKUtil doesn't clean quorum setting properly Key: HBASE-13796 URL: https://issues.apache.org/jira/browse/HBASE-13796 Project: HBase Issue Type: Bug Affects Versions: 1.0.1, 1.1.0, 0.98.12 Reporter: Geoffrey Jacoby Assignee: Geoffrey Jacoby Priority: Minor Attachments: HBASE-13796.patch ZKUtil.getZooKeeperClusterKey is obviously trying to pull out the ZooKeeper quorum setting from the config object and remove several special characters from it. Due to a misplaced parenthesis, however, it's instead running the replace operation on the config setting _name_, HConstants.ZOOKEEPER_QUORUM, and not the config setting itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13779) Calling table.exists() before table.get() end up with an empty Result
[
https://issues.apache.org/jira/browse/HBASE-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi updated HBASE-13779:
Status: Patch Available (was: Open)
Calling table.exists() before table.get() end up with an empty Result
-
Key: HBASE-13779
URL: https://issues.apache.org/jira/browse/HBASE-13779
Project: HBase
Issue Type: Bug
Affects Versions: 1.1.0, 2.0.0, 1.2.0, 0.98.12.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Attachments: HBASE-13779-test.patch, HBASE-13779-v0.patch
If we call exists() before a get() the result returned by the get() will be
empty.
simple test to verify it:
{code}
Put put = new Put(ROW);
put.add(FAMILY, QUALIFIER, VALUE);
table.put(put);
Get get = new Get(ROW);
boolean exist = table.exists(get);
exist = table.exists(get);
assertEquals(true, exist);
Result result = table.get(get);
// this will fail saying that the Result is empty
// if we remove the exist everything is fine
assertEquals(false, result.isEmpty());
assertTrue(Bytes.equals(VALUE, result.getValue(FAMILY, QUALIFIER)));
{code}
if we use a different Get instance for the get everything works
{code}
...
get = new Get(ROW);
Result result = table.get(get);
assertEquals(false, result.isEmpty());
{code}
HTable.exists() set the checkExistenceOnly flag in the Get so that object is
not reusable by a table.get()
{code}
public boolean exists(final Get get) throws IOException {
get.setCheckExistenceOnly(true);
Result r = get(get);
assert r.getExists() != null;
return r.getExists();
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13723) In table.rb scanners are never closed.
[ https://issues.apache.org/jira/browse/HBASE-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563804#comment-14563804 ] Hudson commented on HBASE-13723: FAILURE: Integrated in HBase-1.0 #934 (See [https://builds.apache.org/job/HBase-1.0/934/]) HBASE-13723 In table.rb scanners are never closed. (enis: rev f2f0b086a83de260c7015946129df8aa306c1180) * hbase-shell/src/main/ruby/hbase/table.rb In table.rb scanners are never closed. -- Key: HBASE-13723 URL: https://issues.apache.org/jira/browse/HBASE-13723 Project: HBase Issue Type: Bug Affects Versions: 1.1.0 Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.1 Attachments: 0001-HBASE-13723-In-table.rb-scanners-are-never-closed.patch, HBASE-13723-v0-trunk.txt Looking at table.rb, it seems that scanners are never closed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13723) In table.rb scanners are never closed.
[ https://issues.apache.org/jira/browse/HBASE-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563614#comment-14563614 ] Enis Soztutar commented on HBASE-13723: --- I've pushed the patch to rest of 0.98+ branches. Thanks JM, Lars. In table.rb scanners are never closed. -- Key: HBASE-13723 URL: https://issues.apache.org/jira/browse/HBASE-13723 Project: HBase Issue Type: Bug Affects Versions: 1.1.0 Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.1 Attachments: 0001-HBASE-13723-In-table.rb-scanners-are-never-closed.patch, HBASE-13723-v0-trunk.txt Looking at table.rb, it seems that scanners are never closed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13796) ZKUtil doesn't clean quorum setting properly
[ https://issues.apache.org/jira/browse/HBASE-13796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Geoffrey Jacoby updated HBASE-13796: Status: Patch Available (was: Open) The patch is a small one-line fix that puts the parenthesis in the right place, plus a covering test. ZKUtil doesn't clean quorum setting properly Key: HBASE-13796 URL: https://issues.apache.org/jira/browse/HBASE-13796 Project: HBase Issue Type: Bug Affects Versions: 0.98.12, 1.1.0, 1.0.1 Reporter: Geoffrey Jacoby Assignee: Geoffrey Jacoby Priority: Minor Attachments: HBASE-13796.patch ZKUtil.getZooKeeperClusterKey is obviously trying to pull out the ZooKeeper quorum setting from the config object and remove several special characters from it. Due to a misplaced parenthesis, however, it's instead running the replace operation on the config setting _name_, HConstants.ZOOKEEPER_QUORUM, and not the config setting itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13779) Calling table.exists() before table.get() end up with an empty Result
[
https://issues.apache.org/jira/browse/HBASE-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi updated HBASE-13779:
Attachment: HBASE-13779-v0.patch
v0 reduces the scope to the exists/get only problem.
In case we need to change the Get params we clone the instance.
(I'll open another jira for the scan, once i have understood it)
I found a couple of strange case cloning scan that I have to understand.
there was an asyncPrefetch param not preserved in the Scan(Scan) constructor.
also some tests relies on their Scan object to be changed
{code}
public void testScanMetrics() throws Exception {
...
Scan scan2 = new Scan();
...
scanner = ht.getScanner(scan2);
...
scanner.close();
// closing the scanner will set the metrics.
assertNotNull(scan2.getScanMetrics());
{code}
Calling table.exists() before table.get() end up with an empty Result
-
Key: HBASE-13779
URL: https://issues.apache.org/jira/browse/HBASE-13779
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 1.1.0, 1.2.0, 0.98.12.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Attachments: HBASE-13779-test.patch, HBASE-13779-v0.patch
If we call exists() before a get() the result returned by the get() will be
empty.
simple test to verify it:
{code}
Put put = new Put(ROW);
put.add(FAMILY, QUALIFIER, VALUE);
table.put(put);
Get get = new Get(ROW);
boolean exist = table.exists(get);
exist = table.exists(get);
assertEquals(true, exist);
Result result = table.get(get);
// this will fail saying that the Result is empty
// if we remove the exist everything is fine
assertEquals(false, result.isEmpty());
assertTrue(Bytes.equals(VALUE, result.getValue(FAMILY, QUALIFIER)));
{code}
if we use a different Get instance for the get everything works
{code}
...
get = new Get(ROW);
Result result = table.get(get);
assertEquals(false, result.isEmpty());
{code}
HTable.exists() set the checkExistenceOnly flag in the Get so that object is
not reusable by a table.get()
{code}
public boolean exists(final Get get) throws IOException {
get.setCheckExistenceOnly(true);
Result r = get(get);
assert r.getExists() != null;
return r.getExists();
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12295) Prevent block eviction under us if reads are in progress from the BBs
[ https://issues.apache.org/jira/browse/HBASE-12295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563680#comment-14563680 ] stack commented on HBASE-12295: --- bq. May be we can only pass the type of the block there? That should be possible. Not a big deal. You can't figure whether L1 or L2 looking at the key? bq. When we tried to directly write the cells to the socket as part of the POC things were directly slow. We'll have to dig in on why. You'd think w/ less intermediaries that it would be faster. bq. Making Result carry it is one option, I think you mean the PB result right? Ain't sure (smile). I was thinking in both cases. Result would give you a CellScanner... client or server side. Result cuts at a different dimension from CellBlock though, right? You get a Result per Get and per Scan response (could be full row or partial). CellBlock is an RPC primitive. Thought was that CellBlock could go other places in HBase too to save space (encoding/compression): e.g. in hfileblock or backing a Result. bq. The approach here was to be simple use the existing Payload. When you say Result - will that not be the current way as how we do for non-java clients? Understood. But pulling the rpc controller back into HRegion is a little perverse. HRegion should not have any rpc pollution. Also, we already pervert rpc controller to carry the payload across the server/rpc barrier, an unnatural usage. Then, controller itself is not actually doing any 'controlling' of the rpc -- it is just a place holder in rpc that protobuf actually recommends we avoid (i.e. generate own rpc interface rather than use the one protoc generates). bq. Yes, this will be a zero copy. But we are copying Cells into CellBlock in memory and then we put the CellBlock on the wire? Would be nice if we could write the CellBlock directly on the wire (maybe this is not possible). That said, for now at least, the creation of CellBlock is a good place for triggering decrement of backing block reference. bq. Scans have states and gets do not have states as gets operate with in Region. Scans are Gets. If that is not enough, can we mimic scan context in Get? Would that help? Then we could have one means rather than two keeping account. bq. ...and the CP tries to use those Cells as its state. So, as is, we'll make a copy per registered CP? bq. finalizeScan(boolean finalizeAll). You want a delayed close, one that completes only after all outstanding scanners are done? Can we have scanner close set up one of these? So the boolean does what for compacted files? It says, don't evict files that are being read though they have been compacted? (Is this like the finalizeScan case at all? Where you want to do a delayed close until no more references?) One other thought is how you folks thinking of testing this stuff? Prevent block eviction under us if reads are in progress from the BBs - Key: HBASE-12295 URL: https://issues.apache.org/jira/browse/HBASE-12295 Project: HBase Issue Type: Sub-task Components: regionserver, Scanners Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 2.0.0 Attachments: HBASE-12295.pdf, HBASE-12295_trunk.patch While we try to serve the reads from the BBs directly from the block cache, we need to ensure that the blocks does not get evicted under us while reading. This JIRA is to discuss and implement a strategy for the same. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13796) ZKUtil doesn't clean quorum setting properly
Geoffrey Jacoby created HBASE-13796: --- Summary: ZKUtil doesn't clean quorum setting properly Key: HBASE-13796 URL: https://issues.apache.org/jira/browse/HBASE-13796 Project: HBase Issue Type: Bug Affects Versions: 0.98.12, 1.1.0, 1.0.1 Reporter: Geoffrey Jacoby Assignee: Geoffrey Jacoby Priority: Minor ZKUtil.getZooKeeperClusterKey is obviously trying to pull out the ZooKeeper quorum setting from the config object and remove several special characters from it. Due to a misplaced parenthesis, however, it's instead running the replace operation on the config setting _name_, HConstants.ZOOKEEPER_QUORUM, and not the config setting itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13789) ForeignException should not be sent to the client
[
https://issues.apache.org/jira/browse/HBASE-13789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563751#comment-14563751
]
Hadoop QA commented on HBASE-13789:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12735741/HBASE-13789-v0.patch
against master branch at commit 4aa7209826ae42e47089fd20747d014183fccb6f.
ATTACHMENT ID: 12735741
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14221//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14221//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14221//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14221//console
This message is automatically generated.
ForeignException should not be sent to the client
-
Key: HBASE-13789
URL: https://issues.apache.org/jira/browse/HBASE-13789
Project: HBase
Issue Type: Bug
Components: Client, master
Affects Versions: 2.0.0, 0.98.13, 1.0.1.1, 1.1.0.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Attachments: HBASE-13789-v0.patch
ForeignException is in hbase-server so the client will not be able to
deserialize it, and also it will hide the DoNotRetryException of the cause.
I haven't found an easy way to test it, aside manually looking at the logs.
and this stuff will go away with proc-v2. so for now the easy workaround is
catch the ForeignException in the master which are just the few places
related to proc-v1 and throw the cause to the client
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13797) Fix resource leak in HBaseFsck
Ted Yu created HBASE-13797:
--
Summary: Fix resource leak in HBaseFsck
Key: HBASE-13797
URL: https://issues.apache.org/jira/browse/HBASE-13797
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
In HBaseFsck#unassignMetaReplica() , ZooKeeperWatcher is not closed upon return
from the method:
{code}
ZKUtil.deleteNode(zkw,
zkw.getZNodeForReplica(hi.metaEntry.getReplicaId()));
{code}
This leads to resource leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13798) TestFromClientSide* don't close the Table
Matteo Bertozzi created HBASE-13798: --- Summary: TestFromClientSide* don't close the Table Key: HBASE-13798 URL: https://issues.apache.org/jira/browse/HBASE-13798 Project: HBase Issue Type: Bug Components: test Affects Versions: 1.1.0, 2.0.0 Reporter: Matteo Bertozzi Priority: Trivial TestFromClientSide, TestFromClientSide3 have lots of tests with the Table object not closed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13761) Optimize FuzzyRowFilter
[ https://issues.apache.org/jira/browse/HBASE-13761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563645#comment-14563645 ] Hudson commented on HBASE-13761: SUCCESS: Integrated in HBase-TRUNK #6523 (See [https://builds.apache.org/job/HBase-TRUNK/6523/]) HBASE-13761 Optimize FuzzyRowFilter (Vladimir Rodionov) (ndimiduk: rev f0a1ca4a6f176dcac3e82745ae7d536cbef04d94) * dev-support/test-patch.properties * hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFuzzyRowFilterEndToEnd.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/UnsafeAccess.java * hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFuzzyRowFilter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FuzzyRowFilter.java Optimize FuzzyRowFilter --- Key: HBASE-13761 URL: https://issues.apache.org/jira/browse/HBASE-13761 Project: HBase Issue Type: Improvement Components: Filters Affects Versions: 2.0.0, 1.1.0, 0.98.13 Reporter: Vladimir Rodionov Assignee: Vladimir Rodionov Priority: Minor Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.1 Attachments: 13761-v8.patch, HBASE-13761-0.98.patch, HBASE-13761-branch-1.patch, HBASE-13761.patch, HBASE-13761_2.patch, HBASE-13761_4.patch, HBASE-13761_5.patch, HBASE-13761_6.patch, HBASE-13761_7.patch FuzzyRowFilter has some room for improvements: a lot of byte-by-byte arithmetic, non-efficient algorithm of selecting next candidate row etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13797) Fix resource leak in HBaseFsck
[
https://issues.apache.org/jira/browse/HBASE-13797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-13797:
---
Status: Patch Available (was: Open)
Fix resource leak in HBaseFsck
--
Key: HBASE-13797
URL: https://issues.apache.org/jira/browse/HBASE-13797
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Attachments: 13797-v1.txt
In HBaseFsck#unassignMetaReplica() , ZooKeeperWatcher is not closed upon
return from the method:
{code}
ZKUtil.deleteNode(zkw,
zkw.getZNodeForReplica(hi.metaEntry.getReplicaId()));
{code}
This leads to resource leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13797) Fix resource leak in HBaseFsck
[
https://issues.apache.org/jira/browse/HBASE-13797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-13797:
---
Attachment: 13797-v1.txt
Fix resource leak in HBaseFsck
--
Key: HBASE-13797
URL: https://issues.apache.org/jira/browse/HBASE-13797
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Attachments: 13797-v1.txt
In HBaseFsck#unassignMetaReplica() , ZooKeeperWatcher is not closed upon
return from the method:
{code}
ZKUtil.deleteNode(zkw,
zkw.getZNodeForReplica(hi.metaEntry.getReplicaId()));
{code}
This leads to resource leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13761) Optimize FuzzyRowFilter
[ https://issues.apache.org/jira/browse/HBASE-13761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563686#comment-14563686 ] Hudson commented on HBASE-13761: FAILURE: Integrated in HBase-1.1 #508 (See [https://builds.apache.org/job/HBase-1.1/508/]) HBASE-13761 Optimize FuzzyRowFilter (Vladimir Rodionov) (ndimiduk: rev a1043fb0fabd967eb83ef180e2e747352092d03a) * hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFuzzyRowFilterEndToEnd.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/UnsafeAccess.java * hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFuzzyRowFilter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FuzzyRowFilter.java * dev-support/test-patch.properties Optimize FuzzyRowFilter --- Key: HBASE-13761 URL: https://issues.apache.org/jira/browse/HBASE-13761 Project: HBase Issue Type: Improvement Components: Filters Affects Versions: 2.0.0, 1.1.0, 0.98.13 Reporter: Vladimir Rodionov Assignee: Vladimir Rodionov Priority: Minor Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.1 Attachments: 13761-v8.patch, HBASE-13761-0.98.patch, HBASE-13761-branch-1.patch, HBASE-13761.patch, HBASE-13761_2.patch, HBASE-13761_4.patch, HBASE-13761_5.patch, HBASE-13761_6.patch, HBASE-13761_7.patch FuzzyRowFilter has some room for improvements: a lot of byte-by-byte arithmetic, non-efficient algorithm of selecting next candidate row etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13784) Add Async Client Table API
[ https://issues.apache.org/jira/browse/HBASE-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563727#comment-14563727 ] stack commented on HBASE-13784: --- bq. With the work I am doing I was trying to change as little as possible to the current behaviour. Fair enough. Suggestions for what might be improved upon given your now intimate knowledge of rpc appreciated as follow ons. bq. Should I add an overall timeout to the RetryingResponsePromise? (It seems I forgot to add a stop after max amount of retries has been reached in current patch) Well, we have timeouts and retries now (after you fix the 'stop' missed in current patch?) Adding another timeout in RetryingResponsePromise would be on top of these? Thanks for taking a look at [~louiscryan]'s work. bq. I would like to propose a bit different and simpler Api than is currently implemented in Table. No objection here. We need these? mutate(ListMutation): ResponsePromiseVoid - Will not accept Append and Increment because of nonce requirement. mutate(RowMutations): ResponsePromiseVoid - Will not accept Append and Increment mutate(ListRowMutations): ResponsePromiseVoid - Will not accept Append and Increment Is it just because Append and Increment are not Mutations? Lets fix that rather than do above? How we fix it so you don't need RowMutation and Mutation? Otherwise, all looks good (PromiseKeeping and changing scan... FYI, Scan has had a bunch of work done since you were around last ... did you notice? It should be easier to fit it to your new form that previous). Suggest you float message on dev list to get more input on your new API set. Nice work [~jurmous] Add Async Client Table API -- Key: HBASE-13784 URL: https://issues.apache.org/jira/browse/HBASE-13784 Project: HBase Issue Type: New Feature Reporter: Jurriaan Mous Assignee: Jurriaan Mous Attachments: HBASE-13784-v1.patch, HBASE-13784.patch With the introduction of the Async HBase RPC Client it is possible to create an Async Table API and more. This issue is focussed on creating a first async Table API so it is possible to do any non deprecated Table call in an async way. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13723) In table.rb scanners are never closed.
[ https://issues.apache.org/jira/browse/HBASE-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-13723: -- Fix Version/s: 1.2.0 1.0.2 0.98.14 In table.rb scanners are never closed. -- Key: HBASE-13723 URL: https://issues.apache.org/jira/browse/HBASE-13723 Project: HBase Issue Type: Bug Affects Versions: 1.1.0 Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.1 Attachments: 0001-HBASE-13723-In-table.rb-scanners-are-never-closed.patch, HBASE-13723-v0-trunk.txt Looking at table.rb, it seems that scanners are never closed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13616) Move ServerShutdownHandler to Pv2
[ https://issues.apache.org/jira/browse/HBASE-13616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563767#comment-14563767 ] Elliott Clark commented on HBASE-13616: --- +1 As long as this is passing IT tests. It looks like a great simplification. Move ServerShutdownHandler to Pv2 - Key: HBASE-13616 URL: https://issues.apache.org/jira/browse/HBASE-13616 Project: HBase Issue Type: Sub-task Components: proc-v2 Affects Versions: 1.1.0 Reporter: stack Assignee: stack Attachments: 13616.wip.txt, 13616.wip.v3.branch-1.txt, 13616.wip.v4.branch-1.txt, 13616.wip.v5.branch-1.1.txt, 13616.wip.v6.branch-1.txt, 13616.wip.v7.branch-1.txt, 13616v10.branch-1.txt, 13616v11.branch-1.txt, 13616v12.branch-1.txt, 13616v12.branch-1.txt, 13616v13.branch-1.txt, 13616v13.branch-1.txt, 13616v8.branch-1.txt, 13616v9.branch-1.txt, 13616v9.branch-1.txt, 13616wip.v2.txt Move ServerShutdownHandler to run on ProcedureV2. Need this for DLR to work. See HBASE-13567. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13798) TestFromClientSide* don't close the Table
[ https://issues.apache.org/jira/browse/HBASE-13798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563800#comment-14563800 ] Dima Spivak commented on HBASE-13798: - Want me to knock this out, [~mbertozzi]? TestFromClientSide* don't close the Table - Key: HBASE-13798 URL: https://issues.apache.org/jira/browse/HBASE-13798 Project: HBase Issue Type: Bug Components: test Affects Versions: 2.0.0, 1.1.0 Reporter: Matteo Bertozzi Priority: Trivial TestFromClientSide, TestFromClientSide3 have lots of tests with the Table object not closed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13616) Move ServerShutdownHandler to Pv2
[ https://issues.apache.org/jira/browse/HBASE-13616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563403#comment-14563403 ] stack commented on HBASE-13616: --- javadoc is not from this patch but I can fix on commit. Long lines are from generated code. Any +1s out there? Move ServerShutdownHandler to Pv2 - Key: HBASE-13616 URL: https://issues.apache.org/jira/browse/HBASE-13616 Project: HBase Issue Type: Sub-task Components: proc-v2 Affects Versions: 1.1.0 Reporter: stack Assignee: stack Attachments: 13616.wip.txt, 13616.wip.v3.branch-1.txt, 13616.wip.v4.branch-1.txt, 13616.wip.v5.branch-1.1.txt, 13616.wip.v6.branch-1.txt, 13616.wip.v7.branch-1.txt, 13616v10.branch-1.txt, 13616v11.branch-1.txt, 13616v12.branch-1.txt, 13616v12.branch-1.txt, 13616v13.branch-1.txt, 13616v13.branch-1.txt, 13616v8.branch-1.txt, 13616v9.branch-1.txt, 13616v9.branch-1.txt, 13616wip.v2.txt Move ServerShutdownHandler to run on ProcedureV2. Need this for DLR to work. See HBASE-13567. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13795) Offline regions hard to find, provide highlighting/state info
Lars George created HBASE-13795:
---
Summary: Offline regions hard to find, provide highlighting/state
info
Key: HBASE-13795
URL: https://issues.apache.org/jira/browse/HBASE-13795
Project: HBase
Issue Type: Bug
Components: UI
Affects Versions: 1.1.0
Reporter: Lars George
Fix For: 2.0.0
{{close_region}} removes region from {{rs-status}} page, but does not increase
count in {{master-status}} page. {{table.jsp}} shows all regions but no
indication of their state. In 0.92 we had highlighting of offline regions (red
background). We should do the same, and maybe also add a column that states the
region status.
Note that using {{close_region}} is hackery, but a region could be offline for
other reasons too!
I also suggest adding some (maybe time delayed) check into the master to detect
unassigned regions and show them properly in the table with metrics for each
user table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13761) Optimize FuzzyRowFilter
[ https://issues.apache.org/jira/browse/HBASE-13761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563286#comment-14563286 ] Nick Dimiduk commented on HBASE-13761: -- This looks like a implementation enhancement only, so I think it's good by our semver guidelines to apply on branch-1.1 and branch-1.0. Fine by you [~enis]? Will apply to 0.98 as well [~apurtell]. Let me know if you have objection. Optimize FuzzyRowFilter --- Key: HBASE-13761 URL: https://issues.apache.org/jira/browse/HBASE-13761 Project: HBase Issue Type: Improvement Components: Filters Affects Versions: 2.0.0, 1.1.0, 0.98.13 Reporter: Vladimir Rodionov Assignee: Vladimir Rodionov Priority: Minor Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.1 Attachments: 13761-v8.patch, HBASE-13761-0.98.patch, HBASE-13761-branch-1.patch, HBASE-13761.patch, HBASE-13761_2.patch, HBASE-13761_4.patch, HBASE-13761_5.patch, HBASE-13761_6.patch, HBASE-13761_7.patch FuzzyRowFilter has some room for improvements: a lot of byte-by-byte arithmetic, non-efficient algorithm of selecting next candidate row etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13783) Improve error message Could not seek StoreFileScanner to indicate that issue could be a bad disk
[
https://issues.apache.org/jira/browse/HBASE-13783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563344#comment-14563344
]
Matteo Bertozzi commented on HBASE-13783:
-
I don't think a bad disk is the case of the problem.
a couple of times I ended up with an invalid block magic on the RS, on both
localfs and hdfs.
{code}
Caused by: java.io.IOException: Invalid HFile block magic:
?\x04\x12@\x16\x15x\x8D
at org.apache.hadoop.hbase.io.hfile.BlockType.parse(BlockType.java:153)
at org.apache.hadoop.hbase.io.hfile.BlockType.read(BlockType.java:164)
at org.apache.hadoop.hbase.io.hfile.HFileBlock.init(HFileBlock.java:256)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockDataInternal(HFileBlock.java:1867)
... 28 more
{code}
the hfile had no problem while dumping it with the HFile tool, but randomly the
RS was giving that error.
after some tries I ended up changing seek+read with pread and the problem
didn't show up anymore. but I haven't investigated it too much.
Improve error message Could not seek StoreFileScanner to indicate that
issue could be a bad disk
--
Key: HBASE-13783
URL: https://issues.apache.org/jira/browse/HBASE-13783
Project: HBase
Issue Type: Improvement
Reporter: Apekshit Sharma
Priority: Minor
Attachments: HBASE-13783.patch
Feedback from customers.
Following error could mean that a disk has gone bad. We have seen many users
confused by this error.
java.io.IOException: Could not seek StoreFileScanner[HFileScanner for reader
reader=hdfs:///843914879034b56117dfa6b7f3c8383d/content/b6f5e9961d2d4578aea3917c0637bb7c
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12295) Prevent block eviction under us if reads are in progress from the BBs
[ https://issues.apache.org/jira/browse/HBASE-12295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563359#comment-14563359 ] ramkrishna.s.vasudevan commented on HBASE-12295: bq.returnBlock(BlockCacheKey cacheKey, HFileBlock block) bq.Do we need to return the block too in the above? Won't the key be enough? Ideally yes. But as per our current impl we have a type of block whether it is from L2 or L1 and hence needed the block there. May be we can only pass the type of the block there? That should be possible. Not a big deal. bq.Or, consider that we will want to stream out Cells as they come up out of the server when we implement a streaming Interface on the server. Okie. When we tried to directly write the cells to the socket as part of the POC things were directly slow. May be a different type of protocol/approach may be needed there. bq.Hmm... pulling the CellBlock into the Region from the ipc layer? I have thought that Result should carry CellBlocks This would be an extra copy, right? If we wanted to get to zero copy, would it be possible if we went this route? Yes, this will be a zero copy. Currently while creating cell block there is no copy we do and directly use the encoder to create it. same here except that it is now in HRegion. Making Result carry it is one option, I think you mean the PB result right? The approach here was to be simple use the existing Payload. When you say Result - will that not be the current way as how we do for non-java clients? bq.Nah. You can't pull an oddball RPC datastructure back into HRegion. Could it be done in the Result itself? Same as above. bq.He has added a bunch of accounting on where scan is at... state, and has scans doing heartbeating, and early returns. Can you make use of this work of his? I had a look at it. Will check once more before commenting back. But in our case we need to handle both scans and gets. Scans have states and gets do not have states as gets operate with in Region. bq.Tell us more about the marking of Cells from L2 with a new Interface and why CP need special treatment, need Cells copied when read from CP. We have to do this? CPs are bit tricky. Take a CP which is trying to implement a postScannerOpen hook by wrapping the original scanner. Now in a non CP approach we have the control on the result and the cellblock creation and we are sure that once the cell block is created we no longer refer to the cells from the hfileblocks. But when you have a CP there is a high chance that those cells are referred for a longer time and the CP tries to use those Cells as its state. In those cases, if we think that the blocks ref count can be decremented just because the results have been fetched, we end up corrupting the states of those CPs. Hence we need to do a copy of the result. bq.finalizeScan(boolean finalizeAll). Though we have completed the implementation, we are still seeing if there is a better way,, but I have done some analysis and I fear that may be very very tricky. I can come up with a write up after some more analysis but overall the problem is that the scanner flow has some optimizaitons where we proactively close some of the scanner from the heap just because they don't return any result (infact we nullify them also). In such cases just calling close will not be enough because already those StoreFileScanners could be closed and we will lose the reference to those scanners. Hence thought of adding an explicit API to do it. And added to that for the scan case the close() call alone won't work because there are going to be set of next() calls for a scan to finish and it makes it better if we clear the references of those cells then and there. And in case of scans the latest block would be needed for the subsequent next() calls as Scans are with States. bq. In such a case we don’t evict the block if the ref count 0, instead we mark those blocks with a Boolean. This is a special case. In case of compaction after the files are compacted we know that the compacted files are no longer needed and we forcefully try to evict them from the block cache. But now if there were any parallel scans operating on those files we just cannot evict them. So we use the same ref count mechanism and see if the block can really evicted (even if it is forceful). All such blocks would automatically be evicted once the read operation using that block gets completed. (in the sense on decrementing a 'marked' block to 0 we call evict forcefully). This ensures that the results are not corrupted. Prevent block eviction under us if reads are in progress from the BBs - Key: HBASE-12295 URL: https://issues.apache.org/jira/browse/HBASE-12295 Project: HBase Issue Type:
[jira] [Commented] (HBASE-13794) Add RegionLoad.getStorefiles(Bytes[] family)
[ https://issues.apache.org/jira/browse/HBASE-13794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563280#comment-14563280 ] Jean-Marc Spaggiari commented on HBASE-13794: - This is for manual compactions. On very big clusters (1K RS) with pretty big tables, and few CFs, being able to see how many files we have on a CF for a given region will allow to save a lot of IOs. Today I think this information is available using protobufadmin. Goal is not to keep this information up to date and always available but being able to query that when required. Add RegionLoad.getStorefiles(Bytes[] family) Key: HBASE-13794 URL: https://issues.apache.org/jira/browse/HBASE-13794 Project: HBase Issue Type: Bug Reporter: Jean-Marc Spaggiari RegionLoad already has getStorefiles but it might be very useful to get this information per column familly instead of per region only. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13616) Move ServerShutdownHandler to Pv2
[
https://issues.apache.org/jira/browse/HBASE-13616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563325#comment-14563325
]
Hadoop QA commented on HBASE-13616:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12735880/13616v13.branch-1.txt
against branch-1 branch at commit e9afc9a267b0a8579840145f1dc584fd246d0fbc.
ATTACHMENT ID: 12735880
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 29 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+regionsOnCrashedServer_ =
java.util.Collections.unmodifiableList(regionsOnCrashedServer_);
+ new java.lang.String[] { UserInfo, UnmodifiedTableSchema,
ModifiedTableSchema, DeleteColumnFamilyInModify, });
+ new java.lang.String[] { UserInfo, PreserveSplits,
TableName, TableSchema, RegionInfo, });
+ new java.lang.String[] { ServerName, DistributedLogReplay,
RegionsOnCrashedServer, RegionsToAssign, CarryingMeta, ShouldSplitWal,
});
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14219//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14219//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14219//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14219//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14219//console
This message is automatically generated.
Move ServerShutdownHandler to Pv2
-
Key: HBASE-13616
URL: https://issues.apache.org/jira/browse/HBASE-13616
Project: HBase
Issue Type: Sub-task
Components: proc-v2
Affects Versions: 1.1.0
Reporter: stack
Assignee: stack
Attachments: 13616.wip.txt, 13616.wip.v3.branch-1.txt,
13616.wip.v4.branch-1.txt, 13616.wip.v5.branch-1.1.txt,
13616.wip.v6.branch-1.txt, 13616.wip.v7.branch-1.txt, 13616v10.branch-1.txt,
13616v11.branch-1.txt, 13616v12.branch-1.txt, 13616v12.branch-1.txt,
13616v13.branch-1.txt, 13616v13.branch-1.txt, 13616v8.branch-1.txt,
13616v9.branch-1.txt, 13616v9.branch-1.txt, 13616wip.v2.txt
Move ServerShutdownHandler to run on ProcedureV2. Need this for DLR to work.
See HBASE-13567.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13776) set illegal versions for HColumnDescriptor did not throw IllegalArgumentException
[
https://issues.apache.org/jira/browse/HBASE-13776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563358#comment-14563358
]
Hadoop QA commented on HBASE-13776:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12735881/HBASE-13776.patch
against master branch at commit e9afc9a267b0a8579840145f1dc584fd246d0fbc.
ATTACHMENT ID: 12735881
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14220//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14220//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14220//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14220//console
This message is automatically generated.
set illegal versions for HColumnDescriptor did not throw
IllegalArgumentException
--
Key: HBASE-13776
URL: https://issues.apache.org/jira/browse/HBASE-13776
Project: HBase
Issue Type: Bug
Reporter: Yuhao Bi
Assignee: Yuhao Bi
Attachments: HBASE-13776.patch
HColumnDescriptor hcd = new HColumnDescriptor(
new HColumnDescriptor(HConstants.CATALOG_FAMILY)
.setInMemory(true)
.setScope(HConstants.REPLICATION_SCOPE_LOCAL)
.setBloomFilterType(BloomType.NONE)
.setCacheDataInL1(true));
final int minVersions = 123;
final int maxVersions = 234;
hcd.setMaxVersions(minVersions);
hcd.setMinVersions(maxVersions);
//no exception throw
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13616) Move ServerShutdownHandler to Pv2
[ https://issues.apache.org/jira/browse/HBASE-13616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563576#comment-14563576 ] stack commented on HBASE-13616: --- bq. Is this still having issues on IT tests? These last runs have been good with this patch. I am trying to get to a run size that is larger than what I have done before. The tests take a while. My thought was commit this -- since it is better than what was there before -- and to keep on w/ the scale tests... and if issues, file fixes. Ok w/ you [~eclark]? This is what I am working on. Trying to get to DLR. This is prereq. Move ServerShutdownHandler to Pv2 - Key: HBASE-13616 URL: https://issues.apache.org/jira/browse/HBASE-13616 Project: HBase Issue Type: Sub-task Components: proc-v2 Affects Versions: 1.1.0 Reporter: stack Assignee: stack Attachments: 13616.wip.txt, 13616.wip.v3.branch-1.txt, 13616.wip.v4.branch-1.txt, 13616.wip.v5.branch-1.1.txt, 13616.wip.v6.branch-1.txt, 13616.wip.v7.branch-1.txt, 13616v10.branch-1.txt, 13616v11.branch-1.txt, 13616v12.branch-1.txt, 13616v12.branch-1.txt, 13616v13.branch-1.txt, 13616v13.branch-1.txt, 13616v8.branch-1.txt, 13616v9.branch-1.txt, 13616v9.branch-1.txt, 13616wip.v2.txt Move ServerShutdownHandler to run on ProcedureV2. Need this for DLR to work. See HBASE-13567. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13761) Optimize FuzzyRowFilter
[ https://issues.apache.org/jira/browse/HBASE-13761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563568#comment-14563568 ] Hudson commented on HBASE-13761: SUCCESS: Integrated in HBase-1.2 #110 (See [https://builds.apache.org/job/HBase-1.2/110/]) HBASE-13761 Optimize FuzzyRowFilter (Vladimir Rodionov) (ndimiduk: rev 2f9851af26f5097c880635b18db8be7fa10b7c9e) * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FuzzyRowFilter.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/UnsafeAccess.java * hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFuzzyRowFilterEndToEnd.java * hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFuzzyRowFilter.java * dev-support/test-patch.properties Optimize FuzzyRowFilter --- Key: HBASE-13761 URL: https://issues.apache.org/jira/browse/HBASE-13761 Project: HBase Issue Type: Improvement Components: Filters Affects Versions: 2.0.0, 1.1.0, 0.98.13 Reporter: Vladimir Rodionov Assignee: Vladimir Rodionov Priority: Minor Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.1 Attachments: 13761-v8.patch, HBASE-13761-0.98.patch, HBASE-13761-branch-1.patch, HBASE-13761.patch, HBASE-13761_2.patch, HBASE-13761_4.patch, HBASE-13761_5.patch, HBASE-13761_6.patch, HBASE-13761_7.patch FuzzyRowFilter has some room for improvements: a lot of byte-by-byte arithmetic, non-efficient algorithm of selecting next candidate row etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13761) Optimize FuzzyRowFilter
[ https://issues.apache.org/jira/browse/HBASE-13761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563283#comment-14563283 ] Nick Dimiduk commented on HBASE-13761: -- I'll clean up headers on commit. [~vrodionov] in the future, it'll help to disable auto-comment-formatting in your editor, or to manually omit those unintentional changes from your patches. Makes it hard to tell what's you improving our doc strings and what's the tools making noise. Optimize FuzzyRowFilter --- Key: HBASE-13761 URL: https://issues.apache.org/jira/browse/HBASE-13761 Project: HBase Issue Type: Improvement Components: Filters Affects Versions: 2.0.0, 1.1.0, 0.98.13 Reporter: Vladimir Rodionov Assignee: Vladimir Rodionov Priority: Minor Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.1 Attachments: 13761-v8.patch, HBASE-13761-0.98.patch, HBASE-13761-branch-1.patch, HBASE-13761.patch, HBASE-13761_2.patch, HBASE-13761_4.patch, HBASE-13761_5.patch, HBASE-13761_6.patch, HBASE-13761_7.patch FuzzyRowFilter has some room for improvements: a lot of byte-by-byte arithmetic, non-efficient algorithm of selecting next candidate row etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13776) set illegal versions for HColumnDescriptor did not throw IllegalArgumentException
[ https://issues.apache.org/jira/browse/HBASE-13776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563365#comment-14563365 ] Ted Yu commented on HBASE-13776: lgtm [~byh0831]: Is it possible to add a unit test so that there is no regression in the future ? set illegal versions for HColumnDescriptor did not throw IllegalArgumentException -- Key: HBASE-13776 URL: https://issues.apache.org/jira/browse/HBASE-13776 Project: HBase Issue Type: Bug Reporter: Yuhao Bi Assignee: Yuhao Bi Attachments: HBASE-13776.patch HColumnDescriptor hcd = new HColumnDescriptor( new HColumnDescriptor(HConstants.CATALOG_FAMILY) .setInMemory(true) .setScope(HConstants.REPLICATION_SCOPE_LOCAL) .setBloomFilterType(BloomType.NONE) .setCacheDataInL1(true)); final int minVersions = 123; final int maxVersions = 234; hcd.setMaxVersions(minVersions); hcd.setMinVersions(maxVersions); //no exception throw -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13761) Optimize FuzzyRowFilter
[ https://issues.apache.org/jira/browse/HBASE-13761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-13761: - Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to 0.98+. Thanks for the patch [~vrodionov]. Optimize FuzzyRowFilter --- Key: HBASE-13761 URL: https://issues.apache.org/jira/browse/HBASE-13761 Project: HBase Issue Type: Improvement Components: Filters Affects Versions: 2.0.0, 1.1.0, 0.98.13 Reporter: Vladimir Rodionov Assignee: Vladimir Rodionov Priority: Minor Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.1 Attachments: 13761-v8.patch, HBASE-13761-0.98.patch, HBASE-13761-branch-1.patch, HBASE-13761.patch, HBASE-13761_2.patch, HBASE-13761_4.patch, HBASE-13761_5.patch, HBASE-13761_6.patch, HBASE-13761_7.patch FuzzyRowFilter has some room for improvements: a lot of byte-by-byte arithmetic, non-efficient algorithm of selecting next candidate row etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13776) Setting illegal versions for HColumnDescriptor does not throw IllegalArgumentException
[ https://issues.apache.org/jira/browse/HBASE-13776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-13776: --- Fix Version/s: 1.1.1 1.2.0 1.0.2 0.98.14 2.0.0 Summary: Setting illegal versions for HColumnDescriptor does not throw IllegalArgumentException (was: set illegal versions for HColumnDescriptor did not throw IllegalArgumentException ) Setting illegal versions for HColumnDescriptor does not throw IllegalArgumentException --- Key: HBASE-13776 URL: https://issues.apache.org/jira/browse/HBASE-13776 Project: HBase Issue Type: Bug Reporter: Yuhao Bi Assignee: Yuhao Bi Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.1 Attachments: HBASE-13776.patch HColumnDescriptor hcd = new HColumnDescriptor( new HColumnDescriptor(HConstants.CATALOG_FAMILY) .setInMemory(true) .setScope(HConstants.REPLICATION_SCOPE_LOCAL) .setBloomFilterType(BloomType.NONE) .setCacheDataInL1(true)); final int minVersions = 123; final int maxVersions = 234; hcd.setMaxVersions(minVersions); hcd.setMinVersions(maxVersions); //no exception throw -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13616) Move ServerShutdownHandler to Pv2
[ https://issues.apache.org/jira/browse/HBASE-13616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563414#comment-14563414 ] Matteo Bertozzi commented on HBASE-13616: - looks good to me, +1 Move ServerShutdownHandler to Pv2 - Key: HBASE-13616 URL: https://issues.apache.org/jira/browse/HBASE-13616 Project: HBase Issue Type: Sub-task Components: proc-v2 Affects Versions: 1.1.0 Reporter: stack Assignee: stack Attachments: 13616.wip.txt, 13616.wip.v3.branch-1.txt, 13616.wip.v4.branch-1.txt, 13616.wip.v5.branch-1.1.txt, 13616.wip.v6.branch-1.txt, 13616.wip.v7.branch-1.txt, 13616v10.branch-1.txt, 13616v11.branch-1.txt, 13616v12.branch-1.txt, 13616v12.branch-1.txt, 13616v13.branch-1.txt, 13616v13.branch-1.txt, 13616v8.branch-1.txt, 13616v9.branch-1.txt, 13616v9.branch-1.txt, 13616wip.v2.txt Move ServerShutdownHandler to run on ProcedureV2. Need this for DLR to work. See HBASE-13567. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13616) Move ServerShutdownHandler to Pv2
[ https://issues.apache.org/jira/browse/HBASE-13616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563460#comment-14563460 ] Elliott Clark commented on HBASE-13616: --- Is this still having issues on IT tests? Move ServerShutdownHandler to Pv2 - Key: HBASE-13616 URL: https://issues.apache.org/jira/browse/HBASE-13616 Project: HBase Issue Type: Sub-task Components: proc-v2 Affects Versions: 1.1.0 Reporter: stack Assignee: stack Attachments: 13616.wip.txt, 13616.wip.v3.branch-1.txt, 13616.wip.v4.branch-1.txt, 13616.wip.v5.branch-1.1.txt, 13616.wip.v6.branch-1.txt, 13616.wip.v7.branch-1.txt, 13616v10.branch-1.txt, 13616v11.branch-1.txt, 13616v12.branch-1.txt, 13616v12.branch-1.txt, 13616v13.branch-1.txt, 13616v13.branch-1.txt, 13616v8.branch-1.txt, 13616v9.branch-1.txt, 13616v9.branch-1.txt, 13616wip.v2.txt Move ServerShutdownHandler to run on ProcedureV2. Need this for DLR to work. See HBASE-13567. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13476) Procedure v2 - Add Replay Order logic for child procedures
[ https://issues.apache.org/jira/browse/HBASE-13476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-13476: Resolution: Fixed Status: Resolved (was: Patch Available) Procedure v2 - Add Replay Order logic for child procedures -- Key: HBASE-13476 URL: https://issues.apache.org/jira/browse/HBASE-13476 Project: HBase Issue Type: Sub-task Components: proc-v2 Affects Versions: 2.0.0, 1.1.0 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Fix For: 2.0.0, 1.2.0 Attachments: HBASE-13476-v0.patch, HBASE-13476-v1.patch, HBASE-13476-v2.patch The current replay order logic is only for single-level procedures (which is what we are using today for master operations). To complete the implementation for the notification-bus we need to be able to replay in correct order child procs too. this will not impact the the current procs implementation (create/delete/modify/...) it is just a change at the framework level. https://reviews.apache.org/r/34289/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13761) Optimize FuzzyRowFilter
[ https://issues.apache.org/jira/browse/HBASE-13761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-13761: - Fix Version/s: 1.0.2 Optimize FuzzyRowFilter --- Key: HBASE-13761 URL: https://issues.apache.org/jira/browse/HBASE-13761 Project: HBase Issue Type: Improvement Components: Filters Affects Versions: 2.0.0, 1.1.0, 0.98.13 Reporter: Vladimir Rodionov Assignee: Vladimir Rodionov Priority: Minor Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.1 Attachments: 13761-v8.patch, HBASE-13761-0.98.patch, HBASE-13761-branch-1.patch, HBASE-13761.patch, HBASE-13761_2.patch, HBASE-13761_4.patch, HBASE-13761_5.patch, HBASE-13761_6.patch, HBASE-13761_7.patch FuzzyRowFilter has some room for improvements: a lot of byte-by-byte arithmetic, non-efficient algorithm of selecting next candidate row etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13616) Move ServerShutdownHandler to Pv2
[
https://issues.apache.org/jira/browse/HBASE-13616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563314#comment-14563314
]
Hadoop QA commented on HBASE-13616:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12735880/13616v13.branch-1.txt
against branch-1 branch at commit e9afc9a267b0a8579840145f1dc584fd246d0fbc.
ATTACHMENT ID: 12735880
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 29 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+regionsOnCrashedServer_ =
java.util.Collections.unmodifiableList(regionsOnCrashedServer_);
+ new java.lang.String[] { UserInfo, UnmodifiedTableSchema,
ModifiedTableSchema, DeleteColumnFamilyInModify, });
+ new java.lang.String[] { UserInfo, PreserveSplits,
TableName, TableSchema, RegionInfo, });
+ new java.lang.String[] { ServerName, DistributedLogReplay,
RegionsOnCrashedServer, RegionsToAssign, CarryingMeta, ShouldSplitWal,
});
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14218//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14218//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14218//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14218//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14218//console
This message is automatically generated.
Move ServerShutdownHandler to Pv2
-
Key: HBASE-13616
URL: https://issues.apache.org/jira/browse/HBASE-13616
Project: HBase
Issue Type: Sub-task
Components: proc-v2
Affects Versions: 1.1.0
Reporter: stack
Assignee: stack
Attachments: 13616.wip.txt, 13616.wip.v3.branch-1.txt,
13616.wip.v4.branch-1.txt, 13616.wip.v5.branch-1.1.txt,
13616.wip.v6.branch-1.txt, 13616.wip.v7.branch-1.txt, 13616v10.branch-1.txt,
13616v11.branch-1.txt, 13616v12.branch-1.txt, 13616v12.branch-1.txt,
13616v13.branch-1.txt, 13616v13.branch-1.txt, 13616v8.branch-1.txt,
13616v9.branch-1.txt, 13616v9.branch-1.txt, 13616wip.v2.txt
Move ServerShutdownHandler to run on ProcedureV2. Need this for DLR to work.
See HBASE-13567.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13789) ForeignException should not be sent to the client
[ https://issues.apache.org/jira/browse/HBASE-13789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563350#comment-14563350 ] Jonathan Hsieh commented on HBASE-13789: +1 ForeignException should not be sent to the client - Key: HBASE-13789 URL: https://issues.apache.org/jira/browse/HBASE-13789 Project: HBase Issue Type: Bug Components: Client, master Affects Versions: 2.0.0, 0.98.13, 1.0.1.1, 1.1.0.1 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Attachments: HBASE-13789-v0.patch ForeignException is in hbase-server so the client will not be able to deserialize it, and also it will hide the DoNotRetryException of the cause. I haven't found an easy way to test it, aside manually looking at the logs. and this stuff will go away with proc-v2. so for now the easy workaround is catch the ForeignException in the master which are just the few places related to proc-v1 and throw the cause to the client -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13761) Optimize FuzzyRowFilter
[ https://issues.apache.org/jira/browse/HBASE-13761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563504#comment-14563504 ] Hudson commented on HBASE-13761: FAILURE: Integrated in HBase-1.0 #933 (See [https://builds.apache.org/job/HBase-1.0/933/]) HBASE-13761 Optimize FuzzyRowFilter (Vladimir Rodionov) (ndimiduk: rev 376380ead5ea4a9816777c9301318b282979df71) * hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFuzzyRowFilter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FuzzyRowFilter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFuzzyRowFilterEndToEnd.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/UnsafeAccess.java * dev-support/test-patch.properties Optimize FuzzyRowFilter --- Key: HBASE-13761 URL: https://issues.apache.org/jira/browse/HBASE-13761 Project: HBase Issue Type: Improvement Components: Filters Affects Versions: 2.0.0, 1.1.0, 0.98.13 Reporter: Vladimir Rodionov Assignee: Vladimir Rodionov Priority: Minor Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.1 Attachments: 13761-v8.patch, HBASE-13761-0.98.patch, HBASE-13761-branch-1.patch, HBASE-13761.patch, HBASE-13761_2.patch, HBASE-13761_4.patch, HBASE-13761_5.patch, HBASE-13761_6.patch, HBASE-13761_7.patch FuzzyRowFilter has some room for improvements: a lot of byte-by-byte arithmetic, non-efficient algorithm of selecting next candidate row etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13800) TestStore#testDeleteExpiredStoreFiles should create different data/log directory for each call
[
https://issues.apache.org/jira/browse/HBASE-13800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen Yuan Jiang updated HBASE-13800:
---
Environment: Windows
TestStore#testDeleteExpiredStoreFiles should create different data/log
directory for each call
--
Key: HBASE-13800
URL: https://issues.apache.org/jira/browse/HBASE-13800
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 2.0.0, 1.1.0, 1.2.0
Environment: Windows
Reporter: Stephen Yuan Jiang
Assignee: Stephen Yuan Jiang
Priority: Minor
When TestStore#init() was called twice in
TestStore#testDeleteExpiredStoreFiles, it did not use different base
directory for each call (other tests in the same test suite do). If the
first call did not release the handle of WAL files fast enough, the second
init() call would fail.
This is constantly seen in Windows environment:
{noformat}
java.io.IOException: Target WAL already exists within directory
file:/C:/hbase/hbase-server/target/test-data/f39ecdde-1d04-4332-93c7-4c8df1e08e67/TestStoretestDeleteExpiredStoreFiles/WALs/testDeleteExpiredStoreFiles
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.init(FSHLog.java:525)
at
org.apache.hadoop.hbase.wal.DefaultWALProvider.init(DefaultWALProvider.java:97)
at
org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:147)
at org.apache.hadoop.hbase.wal.WALFactory.init(WALFactory.java:179)
at
org.apache.hadoop.hbase.regionserver.TestStore.init(TestStore.java:185)
at
org.apache.hadoop.hbase.regionserver.TestStore.init(TestStore.java:162)
at
org.apache.hadoop.hbase.regionserver.TestStore.testDeleteExpiredStoreFiles(TestStore.java:307)
at
org.apache.hadoop.hbase.regionserver.TestStore.testDeleteExpiredStoreFiles(TestStore.java:286)
{noformat}
The fix is trivial: just like other tests in the same test suite, use
different base directory for multiple init() calls in the same test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13800) TestStore#testDeleteExpiredStoreFiles should create different data/log directory for each call
[
https://issues.apache.org/jira/browse/HBASE-13800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen Yuan Jiang updated HBASE-13800:
---
Attachment: HBASE-13800.patch
TestStore#testDeleteExpiredStoreFiles should create different data/log
directory for each call
--
Key: HBASE-13800
URL: https://issues.apache.org/jira/browse/HBASE-13800
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 2.0.0, 1.1.0, 1.2.0
Environment: Windows
Reporter: Stephen Yuan Jiang
Assignee: Stephen Yuan Jiang
Priority: Minor
Attachments: HBASE-13800.patch
When TestStore#init() was called twice in
TestStore#testDeleteExpiredStoreFiles, it did not use different base
directory for each call (other tests in the same test suite do). If the
first call did not release the handle of WAL files fast enough, the second
init() call would fail.
This is constantly seen in Windows environment:
{noformat}
java.io.IOException: Target WAL already exists within directory
file:/C:/hbase/hbase-server/target/test-data/f39ecdde-1d04-4332-93c7-4c8df1e08e67/TestStoretestDeleteExpiredStoreFiles/WALs/testDeleteExpiredStoreFiles
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.init(FSHLog.java:525)
at
org.apache.hadoop.hbase.wal.DefaultWALProvider.init(DefaultWALProvider.java:97)
at
org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:147)
at org.apache.hadoop.hbase.wal.WALFactory.init(WALFactory.java:179)
at
org.apache.hadoop.hbase.regionserver.TestStore.init(TestStore.java:185)
at
org.apache.hadoop.hbase.regionserver.TestStore.init(TestStore.java:162)
at
org.apache.hadoop.hbase.regionserver.TestStore.testDeleteExpiredStoreFiles(TestStore.java:307)
at
org.apache.hadoop.hbase.regionserver.TestStore.testDeleteExpiredStoreFiles(TestStore.java:286)
{noformat}
The fix is trivial: just like other tests in the same test suite, use
different base directory for multiple init() calls in the same test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13800) TestStore#testDeleteExpiredStoreFiles should create different data/log directory for each call
[
https://issues.apache.org/jira/browse/HBASE-13800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563922#comment-14563922
]
Ted Yu commented on HBASE-13800:
lgtm
TestStore#testDeleteExpiredStoreFiles should create different data/log
directory for each call
--
Key: HBASE-13800
URL: https://issues.apache.org/jira/browse/HBASE-13800
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 2.0.0, 1.1.0, 1.2.0
Environment: Windows
Reporter: Stephen Yuan Jiang
Assignee: Stephen Yuan Jiang
Priority: Minor
Attachments: HBASE-13800.patch
When TestStore#init() was called twice in
TestStore#testDeleteExpiredStoreFiles, it did not use different base
directory for each call (other tests in the same test suite do). If the
first call did not release the handle of WAL files fast enough, the second
init() call would fail.
This is constantly seen in Windows environment:
{noformat}
java.io.IOException: Target WAL already exists within directory
file:/C:/hbase/hbase-server/target/test-data/f39ecdde-1d04-4332-93c7-4c8df1e08e67/TestStoretestDeleteExpiredStoreFiles/WALs/testDeleteExpiredStoreFiles
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.init(FSHLog.java:525)
at
org.apache.hadoop.hbase.wal.DefaultWALProvider.init(DefaultWALProvider.java:97)
at
org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:147)
at org.apache.hadoop.hbase.wal.WALFactory.init(WALFactory.java:179)
at
org.apache.hadoop.hbase.regionserver.TestStore.init(TestStore.java:185)
at
org.apache.hadoop.hbase.regionserver.TestStore.init(TestStore.java:162)
at
org.apache.hadoop.hbase.regionserver.TestStore.testDeleteExpiredStoreFiles(TestStore.java:307)
at
org.apache.hadoop.hbase.regionserver.TestStore.testDeleteExpiredStoreFiles(TestStore.java:286)
{noformat}
The fix is trivial: just like other tests in the same test suite, use
different base directory for multiple init() calls in the same test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13801) Hadoop src checksum is shown instead of HBase src checksum in master / RS UI
Enis Soztutar created HBASE-13801: - Summary: Hadoop src checksum is shown instead of HBase src checksum in master / RS UI Key: HBASE-13801 URL: https://issues.apache.org/jira/browse/HBASE-13801 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Simple bug. We are showing the Hadoop's source MD5 checksum in the master UI instead of the HBase's one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13718) Add a pretty printed table description to the table detail page of HBase's master
[ https://issues.apache.org/jira/browse/HBASE-13718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-13718: -- Resolution: Fixed Fix Version/s: (was: 1.2.0) Status: Resolved (was: Patch Available) Pushed to master. It didn't apply cleanly to branch-1 or other branches. We can backport on another issue if there's desire. Thanks for the patch Add a pretty printed table description to the table detail page of HBase's master - Key: HBASE-13718 URL: https://issues.apache.org/jira/browse/HBASE-13718 Project: HBase Issue Type: Improvement Components: hbase Affects Versions: 2.0.0 Reporter: Joao Girao Assignee: Joao Girao Priority: Minor Fix For: 2.0.0 Attachments: D38649.diff, D38649.diff.txt, Screen Shot 2015-05-18 at 1.57.50 PM.png HBase's master has an info server that's useful for debugging and getting a general overview of what's in the cluster. It has a page dedicated to describing a cluster. You can reach it by going to something like: http://localhost:54677/table.jsp?name=cluster_test That page currently doesn't have anything about the current table schema. It would be nice to have a table that lists the different column families and how they are set up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13779) Calling table.exists() before table.get() end up with an empty Result
[
https://issues.apache.org/jira/browse/HBASE-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563997#comment-14563997
]
Hadoop QA commented on HBASE-13779:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12735974/HBASE-13779-v0.patch
against master branch at commit 4aa7209826ae42e47089fd20747d014183fccb6f.
ATTACHMENT ID: 12735974
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14224//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14224//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14224//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14224//console
This message is automatically generated.
Calling table.exists() before table.get() end up with an empty Result
-
Key: HBASE-13779
URL: https://issues.apache.org/jira/browse/HBASE-13779
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 1.1.0, 1.2.0, 0.98.12.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Attachments: HBASE-13779-test.patch, HBASE-13779-v0.patch
If we call exists() before a get() the result returned by the get() will be
empty.
simple test to verify it:
{code}
Put put = new Put(ROW);
put.add(FAMILY, QUALIFIER, VALUE);
table.put(put);
Get get = new Get(ROW);
boolean exist = table.exists(get);
exist = table.exists(get);
assertEquals(true, exist);
Result result = table.get(get);
// this will fail saying that the Result is empty
// if we remove the exist everything is fine
assertEquals(false, result.isEmpty());
assertTrue(Bytes.equals(VALUE, result.getValue(FAMILY, QUALIFIER)));
{code}
if we use a different Get instance for the get everything works
{code}
...
get = new Get(ROW);
Result result = table.get(get);
assertEquals(false, result.isEmpty());
{code}
HTable.exists() set the checkExistenceOnly flag in the Get so that object is
not reusable by a table.get()
{code}
public boolean exists(final Get get) throws IOException {
get.setCheckExistenceOnly(true);
Result r = get(get);
assert r.getExists() != null;
return r.getExists();
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13796) ZKUtil doesn't clean quorum setting properly
[ https://issues.apache.org/jira/browse/HBASE-13796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-13796: -- Resolution: Fixed Fix Version/s: 1.2.0 2.0.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to branch-1 and master. Thanks for the patch [~gjacoby] (Took me a while to 'see' what you'd changed) ZKUtil doesn't clean quorum setting properly Key: HBASE-13796 URL: https://issues.apache.org/jira/browse/HBASE-13796 Project: HBase Issue Type: Bug Affects Versions: 1.0.1, 1.1.0, 0.98.12 Reporter: Geoffrey Jacoby Assignee: Geoffrey Jacoby Priority: Minor Fix For: 2.0.0, 1.2.0 Attachments: HBASE-13796.patch ZKUtil.getZooKeeperClusterKey is obviously trying to pull out the ZooKeeper quorum setting from the config object and remove several special characters from it. Due to a misplaced parenthesis, however, it's instead running the replace operation on the config setting _name_, HConstants.ZOOKEEPER_QUORUM, and not the config setting itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13616) Move ServerShutdownHandler to Pv2
[ https://issues.apache.org/jira/browse/HBASE-13616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564154#comment-14564154 ] stack commented on HBASE-13616: --- The two tests fail for others: https://builds.apache.org/view/H-L/view/HBase/job/PreCommit-HBASE-Build/14227/ They pass locally for me. Going to commit. Move ServerShutdownHandler to Pv2 - Key: HBASE-13616 URL: https://issues.apache.org/jira/browse/HBASE-13616 Project: HBase Issue Type: Sub-task Components: proc-v2 Affects Versions: 1.1.0 Reporter: stack Assignee: stack Attachments: 13616.wip.txt, 13616.wip.v3.branch-1.txt, 13616.wip.v4.branch-1.txt, 13616.wip.v5.branch-1.1.txt, 13616.wip.v6.branch-1.txt, 13616.wip.v7.branch-1.txt, 13616v10.branch-1.txt, 13616v11.branch-1.txt, 13616v12.branch-1.txt, 13616v12.branch-1.txt, 13616v13.branch-1.txt, 13616v13.branch-1.txt, 13616v13.txt, 13616v8.branch-1.txt, 13616v9.branch-1.txt, 13616v9.branch-1.txt, 13616wip.v2.txt Move ServerShutdownHandler to run on ProcedureV2. Need this for DLR to work. See HBASE-13567. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13801) Hadoop src checksum is shown instead of HBase src checksum in master / RS UI
[ https://issues.apache.org/jira/browse/HBASE-13801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-13801: -- Attachment: hbase-13801_v1.patch Fixes the issue, and adds ZK version while at it. Hadoop src checksum is shown instead of HBase src checksum in master / RS UI Key: HBASE-13801 URL: https://issues.apache.org/jira/browse/HBASE-13801 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: hbase-13801_v1.patch Simple bug. We are showing the Hadoop's source MD5 checksum in the master UI instead of the HBase's one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13718) Add a pretty printed table description to the table detail page of HBase's master
[ https://issues.apache.org/jira/browse/HBASE-13718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563949#comment-14563949 ] Elliott Clark commented on HBASE-13718: --- +1 I don't think the test failure is related. Add a pretty printed table description to the table detail page of HBase's master - Key: HBASE-13718 URL: https://issues.apache.org/jira/browse/HBASE-13718 Project: HBase Issue Type: Improvement Components: hbase Affects Versions: 2.0.0 Reporter: Joao Girao Assignee: Joao Girao Priority: Minor Fix For: 2.0.0, 1.2.0 Attachments: D38649.diff, D38649.diff.txt, Screen Shot 2015-05-18 at 1.57.50 PM.png HBase's master has an info server that's useful for debugging and getting a general overview of what's in the cluster. It has a page dedicated to describing a cluster. You can reach it by going to something like: http://localhost:54677/table.jsp?name=cluster_test That page currently doesn't have anything about the current table schema. It would be nice to have a table that lists the different column families and how they are set up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13723) In table.rb scanners are never closed.
[ https://issues.apache.org/jira/browse/HBASE-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563944#comment-14563944 ] Hudson commented on HBASE-13723: SUCCESS: Integrated in HBase-1.1 #509 (See [https://builds.apache.org/job/HBase-1.1/509/]) HBASE-13723 In table.rb scanners are never closed. (enis: rev f7023493e29554e59eb6ee4dce5f54314bdc1faf) * hbase-shell/src/main/ruby/hbase/table.rb In table.rb scanners are never closed. -- Key: HBASE-13723 URL: https://issues.apache.org/jira/browse/HBASE-13723 Project: HBase Issue Type: Bug Affects Versions: 1.1.0 Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.1 Attachments: 0001-HBASE-13723-In-table.rb-scanners-are-never-closed.patch, HBASE-13723-v0-trunk.txt Looking at table.rb, it seems that scanners are never closed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13797) Fix resource leak in HBaseFsck
[
https://issues.apache.org/jira/browse/HBASE-13797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564090#comment-14564090
]
Ted Yu commented on HBASE-13797:
{code}
Fetching the console output from the URL
Printing hanging tests
Hanging test : org.apache.hadoop.hbase.rest.TestGetAndPutResource
Printing Failing tests
{code}
The above test is not related to hbck.
Fix resource leak in HBaseFsck
--
Key: HBASE-13797
URL: https://issues.apache.org/jira/browse/HBASE-13797
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Fix For: 2.0.0, 1.2.0, 1.1.1
Attachments: 13797-v1.txt
In HBaseFsck#unassignMetaReplica() , ZooKeeperWatcher is not closed upon
return from the method:
{code}
ZKUtil.deleteNode(zkw,
zkw.getZNodeForReplica(hi.metaEntry.getReplicaId()));
{code}
This leads to resource leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12295) Prevent block eviction under us if reads are in progress from the BBs
[ https://issues.apache.org/jira/browse/HBASE-12295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564140#comment-14564140 ] ramkrishna.s.vasudevan commented on HBASE-12295: bq.You can't figure whether L1 or L2 looking at the key? You mean add some different type of keys for the L1 or L2 case? Can check this. bq.We'll have to dig in on why. You'd think w/ less intermediaries that it would be faster. Yes that was the reason. When we were writing every cell to the socket directly things were really slow. bq.But pulling the rpc controller back into HRegion is a little perverse. Okie, we could see if we can make the REsult carry this cellblock. Will check on this once again. bq.(maybe this is not possible). That said, for now at least, the creation of CellBlock is a good place for triggering decrement of backing block reference. Yes. Right. bq. If that is not enough, can we mimic scan context in Get? Will see on this if it is really possible. bq.So, as is, we'll make a copy per registered CP? Yes but only if the scanner returned from the CP hook is not implementing the new interface. But if we allow the REgionScanner itself to use that new interface and the CP tries to use that new API diligently like how close() would be used, then we are safe we don't need to do that copy. Anyway for non-java case we need to do that copy. bq.You want a delayed close, one that completes only after all outstanding scanners are done? Can we have scanner close set up one of these? Ya, I tried out something yesterday. Seems to work. But checking on all the corner cases, the next patch may have this change. However I doubt on how a scan next() can be handled. It needs some explicit way of decrementing the ref count alone. But generally the call to close() would be an logical end to the ongoing read process including decrementing the ref count on the current list of blocks. bq.So the boolean does what for compacted files? It says, don't evict files that are being read though they have been compacted? (Is this like the finalizeScan case at all? Where you want to do a delayed close until no more references?) A kind of. But this would happen purely internally. Nothing like explicitly calling from the compaciton scanners to set this boolean. Just the block block in the Bucket cache itself would know this and do it. bq.One other thought is how you folks thinking of testing this stuff? Ya as a first measure we have written unit tests to test out these scenarios and ensure we are doing the ref counting correctly. We are adding more tests to it so that we cover the basic scenarios. For the cluster testing we are planning to reduce the bucket cache size and ensure we have frequent eviction scenarios and verify whether any of the results are getting corrupted. Any other suggestion you have here? Prevent block eviction under us if reads are in progress from the BBs - Key: HBASE-12295 URL: https://issues.apache.org/jira/browse/HBASE-12295 Project: HBase Issue Type: Sub-task Components: regionserver, Scanners Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 2.0.0 Attachments: HBASE-12295.pdf, HBASE-12295_trunk.patch While we try to serve the reads from the BBs directly from the block cache, we need to ensure that the blocks does not get evicted under us while reading. This JIRA is to discuss and implement a strategy for the same. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13723) In table.rb scanners are never closed.
[ https://issues.apache.org/jira/browse/HBASE-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564143#comment-14564143 ] Hudson commented on HBASE-13723: SUCCESS: Integrated in HBase-0.98 #1009 (See [https://builds.apache.org/job/HBase-0.98/1009/]) HBASE-13723 In table.rb scanners are never closed. (enis: rev bfbc4b81690fc61aa4035047a71c622139581ba1) * hbase-shell/src/main/ruby/hbase/table.rb In table.rb scanners are never closed. -- Key: HBASE-13723 URL: https://issues.apache.org/jira/browse/HBASE-13723 Project: HBase Issue Type: Bug Affects Versions: 1.1.0 Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.1 Attachments: 0001-HBASE-13723-In-table.rb-scanners-are-never-closed.patch, HBASE-13723-v0-trunk.txt Looking at table.rb, it seems that scanners are never closed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13800) TestStore#testDeleteExpiredStoreFiles should create different data/log directory for each call
Stephen Yuan Jiang created HBASE-13800:
--
Summary: TestStore#testDeleteExpiredStoreFiles should create
different data/log directory for each call
Key: HBASE-13800
URL: https://issues.apache.org/jira/browse/HBASE-13800
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 1.1.0, 2.0.0, 1.2.0
Reporter: Stephen Yuan Jiang
Assignee: Stephen Yuan Jiang
Priority: Minor
When TestStore#init() was called twice in
TestStore#testDeleteExpiredStoreFiles, it did not use different base directory
for each call (other tests in the same test suite do). If the first call did
not release the handle of WAL files fast enough, the second init() call would
fail.
This is constantly seen in Windows environment:
{noformat}
java.io.IOException: Target WAL already exists within directory
file:/C:/hbase/hbase-server/target/test-data/f39ecdde-1d04-4332-93c7-4c8df1e08e67/TestStoretestDeleteExpiredStoreFiles/WALs/testDeleteExpiredStoreFiles
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.init(FSHLog.java:525)
at
org.apache.hadoop.hbase.wal.DefaultWALProvider.init(DefaultWALProvider.java:97)
at
org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:147)
at org.apache.hadoop.hbase.wal.WALFactory.init(WALFactory.java:179)
at
org.apache.hadoop.hbase.regionserver.TestStore.init(TestStore.java:185)
at
org.apache.hadoop.hbase.regionserver.TestStore.init(TestStore.java:162)
at
org.apache.hadoop.hbase.regionserver.TestStore.testDeleteExpiredStoreFiles(TestStore.java:307)
at
org.apache.hadoop.hbase.regionserver.TestStore.testDeleteExpiredStoreFiles(TestStore.java:286)
{noformat}
The fix is trivial: just like other tests in the same test suite, use different
base directory for multiple init() calls in the same test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13616) Move ServerShutdownHandler to Pv2
[ https://issues.apache.org/jira/browse/HBASE-13616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-13616: -- Attachment: 13616v13.txt Pushed to branch-1. Here is a port to master. Let me run it by hadoopqa to see if any differences before I commit. Move ServerShutdownHandler to Pv2 - Key: HBASE-13616 URL: https://issues.apache.org/jira/browse/HBASE-13616 Project: HBase Issue Type: Sub-task Components: proc-v2 Affects Versions: 1.1.0 Reporter: stack Assignee: stack Attachments: 13616.wip.txt, 13616.wip.v3.branch-1.txt, 13616.wip.v4.branch-1.txt, 13616.wip.v5.branch-1.1.txt, 13616.wip.v6.branch-1.txt, 13616.wip.v7.branch-1.txt, 13616v10.branch-1.txt, 13616v11.branch-1.txt, 13616v12.branch-1.txt, 13616v12.branch-1.txt, 13616v13.branch-1.txt, 13616v13.branch-1.txt, 13616v13.txt, 13616v8.branch-1.txt, 13616v9.branch-1.txt, 13616v9.branch-1.txt, 13616wip.v2.txt Move ServerShutdownHandler to run on ProcedureV2. Need this for DLR to work. See HBASE-13567. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13616) Move ServerShutdownHandler to Pv2
[ https://issues.apache.org/jira/browse/HBASE-13616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564017#comment-14564017 ] Hudson commented on HBASE-13616: SUCCESS: Integrated in HBase-1.2 #112 (See [https://builds.apache.org/job/HBase-1.2/112/]) HBASE-13616 Move ServerShutdownHandler to Pv2 (stack: rev 94f0ee7ee391e3fb0ee5f6be7beb8c9cebc4acbc) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerProcedureInterface.java * hbase-server/src/main/java/org/apache/hadoop/hbase/coordination/ZkSplitLogWorkerCoordination.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/coordination/ZKSplitLogManagerCoordination.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManagerOnCluster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableLockManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSHDFSUtils.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestServerCrashProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateTableProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyColumnFamilyProcedure.java * hbase-protocol/src/main/protobuf/MasterProcedure.proto * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestMasterProcedureQueue.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestZKLessAMOnCluster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CreateTableProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureEnv.java * hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureQueue.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TableProcedureInterface.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DeleteTableProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionStates.java * hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestSnapshotClientRetries.java * hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/MetaTableLocator.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DeleteColumnFamilyProcedure.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/AddColumnFamilyProcedure.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/LogReplayHandler.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureTestingUtility.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/DeadServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java * hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterProcedureProtos.java * hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java Move ServerShutdownHandler to Pv2 - Key: HBASE-13616 URL: https://issues.apache.org/jira/browse/HBASE-13616 Project: HBase Issue
[jira] [Commented] (HBASE-13787) use a cli argument parsing library instead of ad-hoc string handling
[ https://issues.apache.org/jira/browse/HBASE-13787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563891#comment-14563891 ] Gabor Liptak commented on HBASE-13787: -- In examples: org.apache.hadoop.hbase.client.example.BufferedMutatorExample uses org.apache.hadoop.util.ToolRunner which uses org.apache.hadoop.util.GenericOptionsParser which uses org.apache.commons.cli.CommandLine org.apache.hadoop.hbase.mapreduce.IndexBuilder uses org.apache.hadoop.util.GenericOptionsParser which uses org.apache.commons.cli.CommandLine, and reads two other parameters directly from args org.apache.hadoop.hbase.mapreduce.SampleUploader uses org.apache.hadoop.util.GenericOptionsParser which uses org.apache.commons.cli.CommandLine, and reads two other parameters directly from args org.apache.hadoop.hbase.thrift.DemoClient parses arguments directly org.apache.hadoop.hbase.thrift.HttpDoAsClient parses arguments directly org.apache.hadoop.hbase.thrift2.DemoClient parses arguments directly use a cli argument parsing library instead of ad-hoc string handling Key: HBASE-13787 URL: https://issues.apache.org/jira/browse/HBASE-13787 Project: HBase Issue Type: Sub-task Components: mapreduce, util Reporter: Sean Busbey several of our mapreduce based tools manually parse strings or rely on system properties for hteir configuration. update all to use the same cli argument parsing library. The particular library used isn't important, but it should be one we already bring in for some other reason. If possible try to make the arguments consistent across all the tools, since you'll be looking broadly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13801) Hadoop src checksum is shown instead of HBase src checksum in master / RS UI
[ https://issues.apache.org/jira/browse/HBASE-13801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-13801: -- Attachment: Screen Shot 2015-05-28 at 4.44.01 PM.png Hadoop src checksum is shown instead of HBase src checksum in master / RS UI Key: HBASE-13801 URL: https://issues.apache.org/jira/browse/HBASE-13801 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: Screen Shot 2015-05-28 at 4.44.01 PM.png, hbase-13801_v1.patch Simple bug. We are showing the Hadoop's source MD5 checksum in the master UI instead of the HBase's one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13787) use a cli argument parsing library instead of ad-hoc string handling
[ https://issues.apache.org/jira/browse/HBASE-13787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564010#comment-14564010 ] Gabor Liptak commented on HBASE-13787: -- org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication parses arguments with callback from ToolRunner org.apache.hadoop.hbase.mapreduce.CellCounter uses org.apache.hadoop.util.GenericOptionsParser which uses org.apache.commons.cli.CommandLine, and reads two other parameters directly from args org.apache.hadoop.hbase.mapreduce.CopyTable parses arguments with callback from ToolRunner org.apache.hadoop.hbase.mapreduce.Driver defers to ProgramDriver to parse org.apache.hadoop.hbase.mapreduce.Export uses org.apache.hadoop.util.GenericOptionsParser which uses org.apache.commons.cli.CommandLine, and reads two other parameters directly from args org.apache.hadoop.hbase.mapreduce.Import uses org.apache.hadoop.util.GenericOptionsParser which uses org.apache.commons.cli.CommandLine, and reads two other parameters directly from args org.apache.hadoop.hbase.mapreduce.ImportTsv uses org.apache.hadoop.util.GenericOptionsParser which uses org.apache.commons.cli.CommandLine, and reads two other parameters directly from args org.apache.commons.cli.LoadIncrementalHFiles uses org.apache.hadoop.util.GenericOptionsParser which uses org.apache.commons.cli.CommandLine, and reads two other parameters directly from args org.apache.commons.cli.RowCounter uses org.apache.hadoop.util.GenericOptionsParser which uses org.apache.commons.cli.CommandLine, and reads two other parameters directly from args org.apache.commons.cli.WALPlayer uses org.apache.hadoop.util.GenericOptionsParser which uses org.apache.commons.cli.CommandLine, and reads two other parameters directly from args use a cli argument parsing library instead of ad-hoc string handling Key: HBASE-13787 URL: https://issues.apache.org/jira/browse/HBASE-13787 Project: HBase Issue Type: Sub-task Components: mapreduce, util Reporter: Sean Busbey several of our mapreduce based tools manually parse strings or rely on system properties for hteir configuration. update all to use the same cli argument parsing library. The particular library used isn't important, but it should be one we already bring in for some other reason. If possible try to make the arguments consistent across all the tools, since you'll be looking broadly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13796) ZKUtil doesn't clean quorum setting properly
[
https://issues.apache.org/jira/browse/HBASE-13796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564056#comment-14564056
]
Hadoop QA commented on HBASE-13796:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12735973/HBASE-13796.patch
against master branch at commit 4aa7209826ae42e47089fd20747d014183fccb6f.
ATTACHMENT ID: 12735973
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14223//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14223//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14223//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14223//console
This message is automatically generated.
ZKUtil doesn't clean quorum setting properly
Key: HBASE-13796
URL: https://issues.apache.org/jira/browse/HBASE-13796
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.1, 1.1.0, 0.98.12
Reporter: Geoffrey Jacoby
Assignee: Geoffrey Jacoby
Priority: Minor
Attachments: HBASE-13796.patch
ZKUtil.getZooKeeperClusterKey is obviously trying to pull out the ZooKeeper
quorum setting from the config object and remove several special characters
from it. Due to a misplaced parenthesis, however, it's instead running the
replace operation on the config setting _name_, HConstants.ZOOKEEPER_QUORUM,
and not the config setting itself.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13797) Fix resource leak in HBaseFsck
[
https://issues.apache.org/jira/browse/HBASE-13797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564058#comment-14564058
]
Hadoop QA commented on HBASE-13797:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12735977/13797-v1.txt
against master branch at commit 4aa7209826ae42e47089fd20747d014183fccb6f.
ATTACHMENT ID: 12735977
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14225//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14225//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14225//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14225//console
This message is automatically generated.
Fix resource leak in HBaseFsck
--
Key: HBASE-13797
URL: https://issues.apache.org/jira/browse/HBASE-13797
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Fix For: 2.0.0, 1.2.0, 1.1.1
Attachments: 13797-v1.txt
In HBaseFsck#unassignMetaReplica() , ZooKeeperWatcher is not closed upon
return from the method:
{code}
ZKUtil.deleteNode(zkw,
zkw.getZNodeForReplica(hi.metaEntry.getReplicaId()));
{code}
This leads to resource leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-13801) Hadoop src checksum is shown instead of HBase src checksum in master / RS UI
[ https://issues.apache.org/jira/browse/HBASE-13801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar resolved HBASE-13801. --- Resolution: Fixed Fix Version/s: 1.1.1 1.2.0 1.0.2 2.0.0 Hadoop Flags: Reviewed Pushed this. Thanks Elliot for taking a look. Hadoop src checksum is shown instead of HBase src checksum in master / RS UI Key: HBASE-13801 URL: https://issues.apache.org/jira/browse/HBASE-13801 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 2.0.0, 1.0.2, 1.2.0, 1.1.1 Attachments: Screen Shot 2015-05-28 at 4.44.01 PM.png, hbase-13801_v1.patch Simple bug. We are showing the Hadoop's source MD5 checksum in the master UI instead of the HBase's one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13800) TestStore#testDeleteExpiredStoreFiles should create different data/log directory for each call
[
https://issues.apache.org/jira/browse/HBASE-13800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564096#comment-14564096
]
Hadoop QA commented on HBASE-13800:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12735995/HBASE-13800.patch
against master branch at commit 4aa7209826ae42e47089fd20747d014183fccb6f.
ATTACHMENT ID: 12735995
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14226//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14226//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14226//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14226//console
This message is automatically generated.
TestStore#testDeleteExpiredStoreFiles should create different data/log
directory for each call
--
Key: HBASE-13800
URL: https://issues.apache.org/jira/browse/HBASE-13800
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 2.0.0, 1.1.0, 1.2.0
Environment: Windows
Reporter: Stephen Yuan Jiang
Assignee: Stephen Yuan Jiang
Priority: Minor
Attachments: HBASE-13800.patch
When TestStore#init() was called twice in
TestStore#testDeleteExpiredStoreFiles, it did not use different base
directory for each call (other tests in the same test suite do). If the
first call did not release the handle of WAL files fast enough, the second
init() call would fail.
This is constantly seen in Windows environment:
{noformat}
java.io.IOException: Target WAL already exists within directory
file:/C:/hbase/hbase-server/target/test-data/f39ecdde-1d04-4332-93c7-4c8df1e08e67/TestStoretestDeleteExpiredStoreFiles/WALs/testDeleteExpiredStoreFiles
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.init(FSHLog.java:525)
at
org.apache.hadoop.hbase.wal.DefaultWALProvider.init(DefaultWALProvider.java:97)
at
org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:147)
at org.apache.hadoop.hbase.wal.WALFactory.init(WALFactory.java:179)
at
org.apache.hadoop.hbase.regionserver.TestStore.init(TestStore.java:185)
at
org.apache.hadoop.hbase.regionserver.TestStore.init(TestStore.java:162)
at
org.apache.hadoop.hbase.regionserver.TestStore.testDeleteExpiredStoreFiles(TestStore.java:307)
at
org.apache.hadoop.hbase.regionserver.TestStore.testDeleteExpiredStoreFiles(TestStore.java:286)
{noformat}
The fix is trivial: just like other tests in the same test suite, use
different base directory for multiple init() calls in the same test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13800) TestStore#testDeleteExpiredStoreFiles should create different data/log directory for each call
[
https://issues.apache.org/jira/browse/HBASE-13800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen Yuan Jiang updated HBASE-13800:
---
Status: Patch Available (was: Open)
TestStore#testDeleteExpiredStoreFiles should create different data/log
directory for each call
--
Key: HBASE-13800
URL: https://issues.apache.org/jira/browse/HBASE-13800
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 1.1.0, 2.0.0, 1.2.0
Environment: Windows
Reporter: Stephen Yuan Jiang
Assignee: Stephen Yuan Jiang
Priority: Minor
Attachments: HBASE-13800.patch
When TestStore#init() was called twice in
TestStore#testDeleteExpiredStoreFiles, it did not use different base
directory for each call (other tests in the same test suite do). If the
first call did not release the handle of WAL files fast enough, the second
init() call would fail.
This is constantly seen in Windows environment:
{noformat}
java.io.IOException: Target WAL already exists within directory
file:/C:/hbase/hbase-server/target/test-data/f39ecdde-1d04-4332-93c7-4c8df1e08e67/TestStoretestDeleteExpiredStoreFiles/WALs/testDeleteExpiredStoreFiles
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.init(FSHLog.java:525)
at
org.apache.hadoop.hbase.wal.DefaultWALProvider.init(DefaultWALProvider.java:97)
at
org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:147)
at org.apache.hadoop.hbase.wal.WALFactory.init(WALFactory.java:179)
at
org.apache.hadoop.hbase.regionserver.TestStore.init(TestStore.java:185)
at
org.apache.hadoop.hbase.regionserver.TestStore.init(TestStore.java:162)
at
org.apache.hadoop.hbase.regionserver.TestStore.testDeleteExpiredStoreFiles(TestStore.java:307)
at
org.apache.hadoop.hbase.regionserver.TestStore.testDeleteExpiredStoreFiles(TestStore.java:286)
{noformat}
The fix is trivial: just like other tests in the same test suite, use
different base directory for multiple init() calls in the same test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13797) Fix resource leak in HBaseFsck
[
https://issues.apache.org/jira/browse/HBASE-13797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-13797:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks for the review, Stephen.
Fix resource leak in HBaseFsck
--
Key: HBASE-13797
URL: https://issues.apache.org/jira/browse/HBASE-13797
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Fix For: 2.0.0, 1.2.0, 1.1.1
Attachments: 13797-v1.txt
In HBaseFsck#unassignMetaReplica() , ZooKeeperWatcher is not closed upon
return from the method:
{code}
ZKUtil.deleteNode(zkw,
zkw.getZNodeForReplica(hi.metaEntry.getReplicaId()));
{code}
This leads to resource leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13800) TestStore#testDeleteExpiredStoreFiles should create unique data/log directory for each call
[
https://issues.apache.org/jira/browse/HBASE-13800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-13800:
---
Resolution: Fixed
Fix Version/s: 1.1.1
1.2.0
2.0.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks for the patch, Stephen.
TestStore#testDeleteExpiredStoreFiles should create unique data/log directory
for each call
---
Key: HBASE-13800
URL: https://issues.apache.org/jira/browse/HBASE-13800
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 2.0.0, 1.1.0, 1.2.0
Environment: Windows
Reporter: Stephen Yuan Jiang
Assignee: Stephen Yuan Jiang
Priority: Minor
Fix For: 2.0.0, 1.2.0, 1.1.1
Attachments: HBASE-13800.patch
When TestStore#init() was called twice in
TestStore#testDeleteExpiredStoreFiles, it did not use different base
directory for each call (other tests in the same test suite do). If the
first call did not release the handle of WAL files fast enough, the second
init() call would fail.
This is constantly seen in Windows environment:
{noformat}
java.io.IOException: Target WAL already exists within directory
file:/C:/hbase/hbase-server/target/test-data/f39ecdde-1d04-4332-93c7-4c8df1e08e67/TestStoretestDeleteExpiredStoreFiles/WALs/testDeleteExpiredStoreFiles
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.init(FSHLog.java:525)
at
org.apache.hadoop.hbase.wal.DefaultWALProvider.init(DefaultWALProvider.java:97)
at
org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:147)
at org.apache.hadoop.hbase.wal.WALFactory.init(WALFactory.java:179)
at
org.apache.hadoop.hbase.regionserver.TestStore.init(TestStore.java:185)
at
org.apache.hadoop.hbase.regionserver.TestStore.init(TestStore.java:162)
at
org.apache.hadoop.hbase.regionserver.TestStore.testDeleteExpiredStoreFiles(TestStore.java:307)
at
org.apache.hadoop.hbase.regionserver.TestStore.testDeleteExpiredStoreFiles(TestStore.java:286)
{noformat}
The fix is trivial: just like other tests in the same test suite, use
different base directory for multiple init() calls in the same test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13787) use a cli argument parsing library instead of ad-hoc string handling
[ https://issues.apache.org/jira/browse/HBASE-13787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564109#comment-14564109 ] Sean Busbey commented on HBASE-13787: - commons-cli sounds like it'll work just fine then. use a cli argument parsing library instead of ad-hoc string handling Key: HBASE-13787 URL: https://issues.apache.org/jira/browse/HBASE-13787 Project: HBase Issue Type: Sub-task Components: mapreduce, util Reporter: Sean Busbey several of our mapreduce based tools manually parse strings or rely on system properties for hteir configuration. update all to use the same cli argument parsing library. The particular library used isn't important, but it should be one we already bring in for some other reason. If possible try to make the arguments consistent across all the tools, since you'll be looking broadly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13718) Add a pretty printed table description to the table detail page of HBase's master
[ https://issues.apache.org/jira/browse/HBASE-13718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564137#comment-14564137 ] Hudson commented on HBASE-13718: FAILURE: Integrated in HBase-TRUNK #6525 (See [https://builds.apache.org/job/HBase-TRUNK/6525/]) HBASE-13718 added columns schema to table description in web view (eclark: rev d16d3afb6065c7e1c08c846b5aae821aeae7e2f1) * hbase-server/src/main/resources/hbase-webapps/master/table.jsp Add a pretty printed table description to the table detail page of HBase's master - Key: HBASE-13718 URL: https://issues.apache.org/jira/browse/HBASE-13718 Project: HBase Issue Type: Improvement Components: hbase Affects Versions: 2.0.0 Reporter: Joao Girao Assignee: Joao Girao Priority: Minor Fix For: 2.0.0 Attachments: D38649.diff, D38649.diff.txt, Screen Shot 2015-05-18 at 1.57.50 PM.png HBase's master has an info server that's useful for debugging and getting a general overview of what's in the cluster. It has a page dedicated to describing a cluster. You can reach it by going to something like: http://localhost:54677/table.jsp?name=cluster_test That page currently doesn't have anything about the current table schema. It would be nice to have a table that lists the different column families and how they are set up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13801) Hadoop src checksum is shown instead of HBase src checksum in master / RS UI
[ https://issues.apache.org/jira/browse/HBASE-13801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563937#comment-14563937 ] Elliott Clark commented on HBASE-13801: --- +1 Hadoop src checksum is shown instead of HBase src checksum in master / RS UI Key: HBASE-13801 URL: https://issues.apache.org/jira/browse/HBASE-13801 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: Screen Shot 2015-05-28 at 4.44.01 PM.png, hbase-13801_v1.patch Simple bug. We are showing the Hadoop's source MD5 checksum in the master UI instead of the HBase's one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13344) Add enforcer rule that matches our JDK support statement
[
https://issues.apache.org/jira/browse/HBASE-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563958#comment-14563958
]
Elliott Clark commented on HBASE-13344:
---
This seems to break building on 1.8
{code}
[INFO] Restricted to JDK 1.7 yet jdk.tools:jdk.tools:jar:1.7:system contains
com/sun/codemodel/internal/ClassType.class targeted to JDK 1.8
[WARNING] Rule 2: org.apache.maven.plugins.enforcer.EnforceBytecodeVersion
failed with message:
HBase has unsupported dependencies.
HBase requires that all dependencies be compiled with version 1.7 or earlier
of the JDK to properly build from source. You appear to be using a newer
dependency. You can use
either mvn -version or mvn enforcer:display-info to verify what version
is active.
Non-release builds can temporarily build with a newer JDK version by setting
the
'compileSource' property (eg. mvn -DcompileSource=1.8 clean package).
Found Banned Dependency: jdk.tools:jdk.tools:jar:1.7
Use 'mvn dependency:tree' to locate the source of the banned dependencies.
{code}
Add enforcer rule that matches our JDK support statement
Key: HBASE-13344
URL: https://issues.apache.org/jira/browse/HBASE-13344
Project: HBase
Issue Type: Improvement
Components: build
Affects Versions: 2.0.0
Reporter: Sean Busbey
Assignee: Matt Warhaftig
Priority: Minor
Labels: beginner, maven
Fix For: 2.0.0
Attachments: HBASE-13344-master.patch, HBASE-13344-master_v2.patch
The [ref guide gives a list of JDKs that we expect our hbase versions to work
with at runtime|http://hbase.apache.org/book.html#basic.prerequisites].
Let's add in the extra-enforcer-rules mojo and start using [the bytecode
version
rule|http://mojo.codehaus.org/extra-enforcer-rules/enforceBytecodeVersion.html]
to make sure that the result of our builds on a given branch won't fail out
because of a misconfigured target jdk version (or a dependency that targets a
later jdk).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13344) Add enforcer rule that matches our JDK support statement
[
https://issues.apache.org/jira/browse/HBASE-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563969#comment-14563969
]
Matt Warhaftig commented on HBASE-13344:
Hi [~eclark], You are having no success even with {{-DcompileSource=1.8}} added
to the build command?
Add enforcer rule that matches our JDK support statement
Key: HBASE-13344
URL: https://issues.apache.org/jira/browse/HBASE-13344
Project: HBase
Issue Type: Improvement
Components: build
Affects Versions: 2.0.0
Reporter: Sean Busbey
Assignee: Matt Warhaftig
Priority: Minor
Labels: beginner, maven
Fix For: 2.0.0
Attachments: HBASE-13344-master.patch, HBASE-13344-master_v2.patch
The [ref guide gives a list of JDKs that we expect our hbase versions to work
with at runtime|http://hbase.apache.org/book.html#basic.prerequisites].
Let's add in the extra-enforcer-rules mojo and start using [the bytecode
version
rule|http://mojo.codehaus.org/extra-enforcer-rules/enforceBytecodeVersion.html]
to make sure that the result of our builds on a given branch won't fail out
because of a misconfigured target jdk version (or a dependency that targets a
later jdk).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13800) TestStore#testDeleteExpiredStoreFiles should create unique data/log directory for each call
[
https://issues.apache.org/jira/browse/HBASE-13800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-13800:
---
Summary: TestStore#testDeleteExpiredStoreFiles should create unique
data/log directory for each call (was: TestStore#testDeleteExpiredStoreFiles
should create different data/log directory for each call)
TestStore#testDeleteExpiredStoreFiles should create unique data/log directory
for each call
---
Key: HBASE-13800
URL: https://issues.apache.org/jira/browse/HBASE-13800
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 2.0.0, 1.1.0, 1.2.0
Environment: Windows
Reporter: Stephen Yuan Jiang
Assignee: Stephen Yuan Jiang
Priority: Minor
Attachments: HBASE-13800.patch
When TestStore#init() was called twice in
TestStore#testDeleteExpiredStoreFiles, it did not use different base
directory for each call (other tests in the same test suite do). If the
first call did not release the handle of WAL files fast enough, the second
init() call would fail.
This is constantly seen in Windows environment:
{noformat}
java.io.IOException: Target WAL already exists within directory
file:/C:/hbase/hbase-server/target/test-data/f39ecdde-1d04-4332-93c7-4c8df1e08e67/TestStoretestDeleteExpiredStoreFiles/WALs/testDeleteExpiredStoreFiles
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.init(FSHLog.java:525)
at
org.apache.hadoop.hbase.wal.DefaultWALProvider.init(DefaultWALProvider.java:97)
at
org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:147)
at org.apache.hadoop.hbase.wal.WALFactory.init(WALFactory.java:179)
at
org.apache.hadoop.hbase.regionserver.TestStore.init(TestStore.java:185)
at
org.apache.hadoop.hbase.regionserver.TestStore.init(TestStore.java:162)
at
org.apache.hadoop.hbase.regionserver.TestStore.testDeleteExpiredStoreFiles(TestStore.java:307)
at
org.apache.hadoop.hbase.regionserver.TestStore.testDeleteExpiredStoreFiles(TestStore.java:286)
{noformat}
The fix is trivial: just like other tests in the same test suite, use
different base directory for multiple init() calls in the same test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13616) Move ServerShutdownHandler to Pv2
[ https://issues.apache.org/jira/browse/HBASE-13616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-13616: -- Resolution: Fixed Fix Version/s: 1.2.0 2.0.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks for reviews [~mbertozzi] and [~eclark] Committed to master and branch-1. Am continuing to test on rig. Move ServerShutdownHandler to Pv2 - Key: HBASE-13616 URL: https://issues.apache.org/jira/browse/HBASE-13616 Project: HBase Issue Type: Sub-task Components: proc-v2 Affects Versions: 1.1.0 Reporter: stack Assignee: stack Fix For: 2.0.0, 1.2.0 Attachments: 13616.wip.txt, 13616.wip.v3.branch-1.txt, 13616.wip.v4.branch-1.txt, 13616.wip.v5.branch-1.1.txt, 13616.wip.v6.branch-1.txt, 13616.wip.v7.branch-1.txt, 13616v10.branch-1.txt, 13616v11.branch-1.txt, 13616v12.branch-1.txt, 13616v12.branch-1.txt, 13616v13.branch-1.txt, 13616v13.branch-1.txt, 13616v13.txt, 13616v8.branch-1.txt, 13616v9.branch-1.txt, 13616v9.branch-1.txt, 13616wip.v2.txt Move ServerShutdownHandler to run on ProcedureV2. Need this for DLR to work. See HBASE-13567. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13476) Procedure v2 - Add Replay Order logic for child procedures
[ https://issues.apache.org/jira/browse/HBASE-13476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563921#comment-14563921 ] Hudson commented on HBASE-13476: SUCCESS: Integrated in HBase-TRUNK #6524 (See [https://builds.apache.org/job/HBase-TRUNK/6524/]) HBASE-13476 Procedure v2 - Add Replay Order logic for child procedures (matteo.bertozzi: rev 4aa7209826ae42e47089fd20747d014183fccb6f) * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureRecovery.java * hbase-server/src/test/resources/hbase-site.xml * hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureConstants.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureReplayOrder.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/ProcedureWALFormat.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/wal/TestWALProcedureStore.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/ProcedureStore.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/ProcedureTestingUtility.java * hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/ProcedureWALFormatReader.java * hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureExecution.java Procedure v2 - Add Replay Order logic for child procedures -- Key: HBASE-13476 URL: https://issues.apache.org/jira/browse/HBASE-13476 Project: HBase Issue Type: Sub-task Components: proc-v2 Affects Versions: 2.0.0, 1.1.0 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Fix For: 2.0.0, 1.2.0 Attachments: HBASE-13476-v0.patch, HBASE-13476-v1.patch, HBASE-13476-v2.patch The current replay order logic is only for single-level procedures (which is what we are using today for master operations). To complete the implementation for the notification-bus we need to be able to replay in correct order child procs too. this will not impact the the current procs implementation (create/delete/modify/...) it is just a change at the framework level. https://reviews.apache.org/r/34289/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13616) Move ServerShutdownHandler to Pv2
[
https://issues.apache.org/jira/browse/HBASE-13616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564138#comment-14564138
]
Hadoop QA commented on HBASE-13616:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12736005/13616v13.txt
against master branch at commit d16d3afb6065c7e1c08c846b5aae821aeae7e2f1.
ATTACHMENT ID: 12736005
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 23 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+regionsOnCrashedServer_ =
java.util.Collections.unmodifiableList(regionsOnCrashedServer_);
+ new java.lang.String[] { UserInfo, UnmodifiedTableSchema,
ModifiedTableSchema, DeleteColumnFamilyInModify, });
+ new java.lang.String[] { UserInfo, PreserveSplits,
TableName, TableSchema, RegionInfo, });
+ new java.lang.String[] { ServerName, DistributedLogReplay,
RegionsOnCrashedServer, RegionsToAssign, CarryingMeta, ShouldSplitWal,
});
+// TODO Needed? ListString nodes = ZKUtil.listChildrenNoWatch(watcher,
watcher.assignmentZNode);
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestImportExport
org.apache.hadoop.hbase.util.TestProcessBasedCluster
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14227//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14227//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14227//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14227//console
This message is automatically generated.
Move ServerShutdownHandler to Pv2
-
Key: HBASE-13616
URL: https://issues.apache.org/jira/browse/HBASE-13616
Project: HBase
Issue Type: Sub-task
Components: proc-v2
Affects Versions: 1.1.0
Reporter: stack
Assignee: stack
Attachments: 13616.wip.txt, 13616.wip.v3.branch-1.txt,
13616.wip.v4.branch-1.txt, 13616.wip.v5.branch-1.1.txt,
13616.wip.v6.branch-1.txt, 13616.wip.v7.branch-1.txt, 13616v10.branch-1.txt,
13616v11.branch-1.txt, 13616v12.branch-1.txt, 13616v12.branch-1.txt,
13616v13.branch-1.txt, 13616v13.branch-1.txt, 13616v13.txt,
13616v8.branch-1.txt, 13616v9.branch-1.txt, 13616v9.branch-1.txt,
13616wip.v2.txt
Move ServerShutdownHandler to run on ProcedureV2. Need this for DLR to work.
See HBASE-13567.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13779) Calling table.exists() before table.get() end up with an empty Result
[
https://issues.apache.org/jira/browse/HBASE-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-13779:
--
Attachment: HBASE-13779-v0.patch
Calling table.exists() before table.get() end up with an empty Result
-
Key: HBASE-13779
URL: https://issues.apache.org/jira/browse/HBASE-13779
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 1.1.0, 1.2.0, 0.98.12.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Attachments: HBASE-13779-test.patch, HBASE-13779-v0.patch,
HBASE-13779-v0.patch
If we call exists() before a get() the result returned by the get() will be
empty.
simple test to verify it:
{code}
Put put = new Put(ROW);
put.add(FAMILY, QUALIFIER, VALUE);
table.put(put);
Get get = new Get(ROW);
boolean exist = table.exists(get);
exist = table.exists(get);
assertEquals(true, exist);
Result result = table.get(get);
// this will fail saying that the Result is empty
// if we remove the exist everything is fine
assertEquals(false, result.isEmpty());
assertTrue(Bytes.equals(VALUE, result.getValue(FAMILY, QUALIFIER)));
{code}
if we use a different Get instance for the get everything works
{code}
...
get = new Get(ROW);
Result result = table.get(get);
assertEquals(false, result.isEmpty());
{code}
HTable.exists() set the checkExistenceOnly flag in the Get so that object is
not reusable by a table.get()
{code}
public boolean exists(final Get get) throws IOException {
get.setCheckExistenceOnly(true);
Result r = get(get);
assert r.getExists() != null;
return r.getExists();
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13779) Calling table.exists() before table.get() end up with an empty Result
[
https://issues.apache.org/jira/browse/HBASE-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564139#comment-14564139
]
stack commented on HBASE-13779:
---
+1
Hanging test is
kalashnikov:hbase.git.commit stack$ python ./dev-support/findHangingTests.py
https://builds.apache.org/job/PreCommit-HBASE-Build/14224//console
Fetching the console output from the URL
Printing hanging tests
Hanging test : org.apache.hadoop.hbase.io.hfile.TestHFileEncryption
Printing Failing tests
Which seems unrelated. Let me rerun just in case.
Calling table.exists() before table.get() end up with an empty Result
-
Key: HBASE-13779
URL: https://issues.apache.org/jira/browse/HBASE-13779
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 1.1.0, 1.2.0, 0.98.12.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Attachments: HBASE-13779-test.patch, HBASE-13779-v0.patch
If we call exists() before a get() the result returned by the get() will be
empty.
simple test to verify it:
{code}
Put put = new Put(ROW);
put.add(FAMILY, QUALIFIER, VALUE);
table.put(put);
Get get = new Get(ROW);
boolean exist = table.exists(get);
exist = table.exists(get);
assertEquals(true, exist);
Result result = table.get(get);
// this will fail saying that the Result is empty
// if we remove the exist everything is fine
assertEquals(false, result.isEmpty());
assertTrue(Bytes.equals(VALUE, result.getValue(FAMILY, QUALIFIER)));
{code}
if we use a different Get instance for the get everything works
{code}
...
get = new Get(ROW);
Result result = table.get(get);
assertEquals(false, result.isEmpty());
{code}
HTable.exists() set the checkExistenceOnly flag in the Get so that object is
not reusable by a table.get()
{code}
public boolean exists(final Get get) throws IOException {
get.setCheckExistenceOnly(true);
Result r = get(get);
assert r.getExists() != null;
return r.getExists();
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13801) Hadoop src checksum is shown instead of HBase src checksum in master / RS UI
[ https://issues.apache.org/jira/browse/HBASE-13801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564165#comment-14564165 ] Hudson commented on HBASE-13801: FAILURE: Integrated in HBase-1.0 #935 (See [https://builds.apache.org/job/HBase-1.0/935/]) HBASE-13801 Hadoop src checksum is shown instead of HBase src checksum in master / RS UI (enis: rev 48a612a89a38e6a86f4020e7e5ddadb404989d71) * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RSStatusTmpl.jamon Hadoop src checksum is shown instead of HBase src checksum in master / RS UI Key: HBASE-13801 URL: https://issues.apache.org/jira/browse/HBASE-13801 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 2.0.0, 1.0.2, 1.2.0, 1.1.1 Attachments: Screen Shot 2015-05-28 at 4.44.01 PM.png, hbase-13801_v1.patch Simple bug. We are showing the Hadoop's source MD5 checksum in the master UI instead of the HBase's one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13723) In table.rb scanners are never closed.
[ https://issues.apache.org/jira/browse/HBASE-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564176#comment-14564176 ] Hudson commented on HBASE-13723: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #960 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/960/]) HBASE-13723 In table.rb scanners are never closed. (enis: rev bfbc4b81690fc61aa4035047a71c622139581ba1) * hbase-shell/src/main/ruby/hbase/table.rb In table.rb scanners are never closed. -- Key: HBASE-13723 URL: https://issues.apache.org/jira/browse/HBASE-13723 Project: HBase Issue Type: Bug Affects Versions: 1.1.0 Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.1 Attachments: 0001-HBASE-13723-In-table.rb-scanners-are-never-closed.patch, HBASE-13723-v0-trunk.txt Looking at table.rb, it seems that scanners are never closed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12295) Prevent block eviction under us if reads are in progress from the BBs
[ https://issues.apache.org/jira/browse/HBASE-12295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564184#comment-14564184 ] Anoop Sam John commented on HBASE-12295: bq.So, as is, we'll make a copy per registered CP? Not really. We will make copy once. Basically the scan/get we have distiguished as 2 path. One in which there is no need for copy and other needs copy. The second case uses the old API itself. Here once we get the cells, we will see whether it is marked using the new Interface, if so we will do copy to a new Cell. And then this new cell is going to be passed to CPs as well as HRS and above layers. Am I making it clear now? Prevent block eviction under us if reads are in progress from the BBs - Key: HBASE-12295 URL: https://issues.apache.org/jira/browse/HBASE-12295 Project: HBase Issue Type: Sub-task Components: regionserver, Scanners Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 2.0.0 Attachments: HBASE-12295.pdf, HBASE-12295_trunk.patch While we try to serve the reads from the BBs directly from the block cache, we need to ensure that the blocks does not get evicted under us while reading. This JIRA is to discuss and implement a strategy for the same. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13779) Calling table.exists() before table.get() end up with an empty Result
[
https://issues.apache.org/jira/browse/HBASE-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564212#comment-14564212
]
Matteo Bertozzi commented on HBASE-13779:
-
{quote}Any reason why not directly making new Get(get) but using
Reflection?{quote}
for compatibility, I wasn't sure if we may have cases where someone extends
Get, so I preferred to go in a safe direction.
Calling table.exists() before table.get() end up with an empty Result
-
Key: HBASE-13779
URL: https://issues.apache.org/jira/browse/HBASE-13779
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 1.1.0, 1.2.0, 0.98.12.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Attachments: HBASE-13779-test.patch, HBASE-13779-v0.patch,
HBASE-13779-v0.patch
If we call exists() before a get() the result returned by the get() will be
empty.
simple test to verify it:
{code}
Put put = new Put(ROW);
put.add(FAMILY, QUALIFIER, VALUE);
table.put(put);
Get get = new Get(ROW);
boolean exist = table.exists(get);
exist = table.exists(get);
assertEquals(true, exist);
Result result = table.get(get);
// this will fail saying that the Result is empty
// if we remove the exist everything is fine
assertEquals(false, result.isEmpty());
assertTrue(Bytes.equals(VALUE, result.getValue(FAMILY, QUALIFIER)));
{code}
if we use a different Get instance for the get everything works
{code}
...
get = new Get(ROW);
Result result = table.get(get);
assertEquals(false, result.isEmpty());
{code}
HTable.exists() set the checkExistenceOnly flag in the Get so that object is
not reusable by a table.get()
{code}
public boolean exists(final Get get) throws IOException {
get.setCheckExistenceOnly(true);
Result r = get(get);
assert r.getExists() != null;
return r.getExists();
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13800) TestStore#testDeleteExpiredStoreFiles should create unique data/log directory for each call
[
https://issues.apache.org/jira/browse/HBASE-13800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564223#comment-14564223
]
Hudson commented on HBASE-13800:
FAILURE: Integrated in HBase-TRUNK #6526 (See
[https://builds.apache.org/job/HBase-TRUNK/6526/])
HBASE-13800 TestStore#testDeleteExpiredStoreFiles should create unique data/log
directory for each call (Stephen Jiang) (tedyu: rev
63d617a0ccabd6bc4c921f7b70a3ff3f83a245b1)
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java
TestStore#testDeleteExpiredStoreFiles should create unique data/log directory
for each call
---
Key: HBASE-13800
URL: https://issues.apache.org/jira/browse/HBASE-13800
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 2.0.0, 1.1.0, 1.2.0
Environment: Windows
Reporter: Stephen Yuan Jiang
Assignee: Stephen Yuan Jiang
Priority: Minor
Fix For: 2.0.0, 1.2.0, 1.1.1
Attachments: HBASE-13800.patch
When TestStore#init() was called twice in
TestStore#testDeleteExpiredStoreFiles, it did not use different base
directory for each call (other tests in the same test suite do). If the
first call did not release the handle of WAL files fast enough, the second
init() call would fail.
This is constantly seen in Windows environment:
{noformat}
java.io.IOException: Target WAL already exists within directory
file:/C:/hbase/hbase-server/target/test-data/f39ecdde-1d04-4332-93c7-4c8df1e08e67/TestStoretestDeleteExpiredStoreFiles/WALs/testDeleteExpiredStoreFiles
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.init(FSHLog.java:525)
at
org.apache.hadoop.hbase.wal.DefaultWALProvider.init(DefaultWALProvider.java:97)
at
org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:147)
at org.apache.hadoop.hbase.wal.WALFactory.init(WALFactory.java:179)
at
org.apache.hadoop.hbase.regionserver.TestStore.init(TestStore.java:185)
at
org.apache.hadoop.hbase.regionserver.TestStore.init(TestStore.java:162)
at
org.apache.hadoop.hbase.regionserver.TestStore.testDeleteExpiredStoreFiles(TestStore.java:307)
at
org.apache.hadoop.hbase.regionserver.TestStore.testDeleteExpiredStoreFiles(TestStore.java:286)
{noformat}
The fix is trivial: just like other tests in the same test suite, use
different base directory for multiple init() calls in the same test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13797) Fix resource leak in HBaseFsck
[
https://issues.apache.org/jira/browse/HBASE-13797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564225#comment-14564225
]
Hudson commented on HBASE-13797:
FAILURE: Integrated in HBase-TRUNK #6526 (See
[https://builds.apache.org/job/HBase-TRUNK/6526/])
HBASE-13797 Fix resource leak in HBaseFsck (tedyu: rev
365ddfaf58821a8fcf22b536d1b6c3b7d886d295)
* hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
Fix resource leak in HBaseFsck
--
Key: HBASE-13797
URL: https://issues.apache.org/jira/browse/HBASE-13797
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Fix For: 2.0.0, 1.2.0, 1.1.1
Attachments: 13797-v1.txt
In HBaseFsck#unassignMetaReplica() , ZooKeeperWatcher is not closed upon
return from the method:
{code}
ZKUtil.deleteNode(zkw,
zkw.getZNodeForReplica(hi.metaEntry.getReplicaId()));
{code}
This leads to resource leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13776) Setting illegal versions for HColumnDescriptor does not throw IllegalArgumentException
[ https://issues.apache.org/jira/browse/HBASE-13776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] AnSec.Biyuhao updated HBASE-13776: -- Status: Open (was: Patch Available) Setting illegal versions for HColumnDescriptor does not throw IllegalArgumentException --- Key: HBASE-13776 URL: https://issues.apache.org/jira/browse/HBASE-13776 Project: HBase Issue Type: Bug Reporter: AnSec.Biyuhao Assignee: AnSec.Biyuhao Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.1 Attachments: HBASE-13776.patch HColumnDescriptor hcd = new HColumnDescriptor( new HColumnDescriptor(HConstants.CATALOG_FAMILY) .setInMemory(true) .setScope(HConstants.REPLICATION_SCOPE_LOCAL) .setBloomFilterType(BloomType.NONE) .setCacheDataInL1(true)); final int minVersions = 123; final int maxVersions = 234; hcd.setMaxVersions(minVersions); hcd.setMinVersions(maxVersions); //no exception throw -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13801) Hadoop src checksum is shown instead of HBase src checksum in master / RS UI
[ https://issues.apache.org/jira/browse/HBASE-13801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564224#comment-14564224 ] Hudson commented on HBASE-13801: FAILURE: Integrated in HBase-TRUNK #6526 (See [https://builds.apache.org/job/HBase-TRUNK/6526/]) HBASE-13801 Hadoop src checksum is shown instead of HBase src checksum in master / RS UI (enis: rev eea28a334cbcc53bd65585943b7cf7d238a598ad) * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RSStatusTmpl.jamon * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon Hadoop src checksum is shown instead of HBase src checksum in master / RS UI Key: HBASE-13801 URL: https://issues.apache.org/jira/browse/HBASE-13801 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 2.0.0, 1.0.2, 1.2.0, 1.1.1 Attachments: Screen Shot 2015-05-28 at 4.44.01 PM.png, hbase-13801_v1.patch Simple bug. We are showing the Hadoop's source MD5 checksum in the master UI instead of the HBase's one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13801) Hadoop src checksum is shown instead of HBase src checksum in master / RS UI
[ https://issues.apache.org/jira/browse/HBASE-13801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564160#comment-14564160 ] Hudson commented on HBASE-13801: SUCCESS: Integrated in HBase-1.2 #113 (See [https://builds.apache.org/job/HBase-1.2/113/]) HBASE-13801 Hadoop src checksum is shown instead of HBase src checksum in master / RS UI (enis: rev 0c90231374ef4265f68def3eab5d71c352fffb97) * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon * hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RSStatusTmpl.jamon Hadoop src checksum is shown instead of HBase src checksum in master / RS UI Key: HBASE-13801 URL: https://issues.apache.org/jira/browse/HBASE-13801 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 2.0.0, 1.0.2, 1.2.0, 1.1.1 Attachments: Screen Shot 2015-05-28 at 4.44.01 PM.png, hbase-13801_v1.patch Simple bug. We are showing the Hadoop's source MD5 checksum in the master UI instead of the HBase's one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13797) Fix resource leak in HBaseFsck
[
https://issues.apache.org/jira/browse/HBASE-13797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564235#comment-14564235
]
Hudson commented on HBASE-13797:
FAILURE: Integrated in HBase-1.2 #114 (See
[https://builds.apache.org/job/HBase-1.2/114/])
HBASE-13797 Fix resource leak in HBaseFsck (tedyu: rev
4545420d611735b8f3e85434d95aafa8feaf86e7)
* hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
Fix resource leak in HBaseFsck
--
Key: HBASE-13797
URL: https://issues.apache.org/jira/browse/HBASE-13797
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Fix For: 2.0.0, 1.2.0, 1.1.1
Attachments: 13797-v1.txt
In HBaseFsck#unassignMetaReplica() , ZooKeeperWatcher is not closed upon
return from the method:
{code}
ZKUtil.deleteNode(zkw,
zkw.getZNodeForReplica(hi.metaEntry.getReplicaId()));
{code}
This leads to resource leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13800) TestStore#testDeleteExpiredStoreFiles should create unique data/log directory for each call
[
https://issues.apache.org/jira/browse/HBASE-13800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564234#comment-14564234
]
Hudson commented on HBASE-13800:
FAILURE: Integrated in HBase-1.2 #114 (See
[https://builds.apache.org/job/HBase-1.2/114/])
HBASE-13800 TestStore#testDeleteExpiredStoreFiles should create unique data/log
directory for each call (Stephen Jiang) (tedyu: rev
423499fbfba5c776b456fbf9cd3762428e1db3ba)
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java
TestStore#testDeleteExpiredStoreFiles should create unique data/log directory
for each call
---
Key: HBASE-13800
URL: https://issues.apache.org/jira/browse/HBASE-13800
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 2.0.0, 1.1.0, 1.2.0
Environment: Windows
Reporter: Stephen Yuan Jiang
Assignee: Stephen Yuan Jiang
Priority: Minor
Fix For: 2.0.0, 1.2.0, 1.1.1
Attachments: HBASE-13800.patch
When TestStore#init() was called twice in
TestStore#testDeleteExpiredStoreFiles, it did not use different base
directory for each call (other tests in the same test suite do). If the
first call did not release the handle of WAL files fast enough, the second
init() call would fail.
This is constantly seen in Windows environment:
{noformat}
java.io.IOException: Target WAL already exists within directory
file:/C:/hbase/hbase-server/target/test-data/f39ecdde-1d04-4332-93c7-4c8df1e08e67/TestStoretestDeleteExpiredStoreFiles/WALs/testDeleteExpiredStoreFiles
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.init(FSHLog.java:525)
at
org.apache.hadoop.hbase.wal.DefaultWALProvider.init(DefaultWALProvider.java:97)
at
org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:147)
at org.apache.hadoop.hbase.wal.WALFactory.init(WALFactory.java:179)
at
org.apache.hadoop.hbase.regionserver.TestStore.init(TestStore.java:185)
at
org.apache.hadoop.hbase.regionserver.TestStore.init(TestStore.java:162)
at
org.apache.hadoop.hbase.regionserver.TestStore.testDeleteExpiredStoreFiles(TestStore.java:307)
at
org.apache.hadoop.hbase.regionserver.TestStore.testDeleteExpiredStoreFiles(TestStore.java:286)
{noformat}
The fix is trivial: just like other tests in the same test suite, use
different base directory for multiple init() calls in the same test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13779) Calling table.exists() before table.get() end up with an empty Result
[
https://issues.apache.org/jira/browse/HBASE-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564241#comment-14564241
]
Hadoop QA commented on HBASE-13779:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12736038/HBASE-13779-v0.patch
against master branch at commit e61bf1bf2582cad20c54585ceea21ec090984c1a.
ATTACHMENT ID: 12736038
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestReplicasClient
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14229//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14229//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14229//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14229//console
This message is automatically generated.
Calling table.exists() before table.get() end up with an empty Result
-
Key: HBASE-13779
URL: https://issues.apache.org/jira/browse/HBASE-13779
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 1.1.0, 1.2.0, 0.98.12.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Attachments: HBASE-13779-test.patch, HBASE-13779-v0.patch,
HBASE-13779-v0.patch
If we call exists() before a get() the result returned by the get() will be
empty.
simple test to verify it:
{code}
Put put = new Put(ROW);
put.add(FAMILY, QUALIFIER, VALUE);
table.put(put);
Get get = new Get(ROW);
boolean exist = table.exists(get);
exist = table.exists(get);
assertEquals(true, exist);
Result result = table.get(get);
// this will fail saying that the Result is empty
// if we remove the exist everything is fine
assertEquals(false, result.isEmpty());
assertTrue(Bytes.equals(VALUE, result.getValue(FAMILY, QUALIFIER)));
{code}
if we use a different Get instance for the get everything works
{code}
...
get = new Get(ROW);
Result result = table.get(get);
assertEquals(false, result.isEmpty());
{code}
HTable.exists() set the checkExistenceOnly flag in the Get so that object is
not reusable by a table.get()
{code}
public boolean exists(final Get get) throws IOException {
get.setCheckExistenceOnly(true);
Result r = get(get);
assert r.getExists() != null;
return r.getExists();
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13779) Calling table.exists() before table.get() end up with an empty Result
[
https://issues.apache.org/jira/browse/HBASE-13779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564177#comment-14564177
]
Anoop Sam John commented on HBASE-13779:
Patch looks good.
bq. get = ReflectionUtils.newInstance(get.getClass(), get);
Any reason why not directly making new Get(get) but using Reflection?
Yes Scan we can not solve because of the scan metric thing. HBASE-1774. I
see another issue we raised to document it. Fine.
Calling table.exists() before table.get() end up with an empty Result
-
Key: HBASE-13779
URL: https://issues.apache.org/jira/browse/HBASE-13779
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 1.1.0, 1.2.0, 0.98.12.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Attachments: HBASE-13779-test.patch, HBASE-13779-v0.patch,
HBASE-13779-v0.patch
If we call exists() before a get() the result returned by the get() will be
empty.
simple test to verify it:
{code}
Put put = new Put(ROW);
put.add(FAMILY, QUALIFIER, VALUE);
table.put(put);
Get get = new Get(ROW);
boolean exist = table.exists(get);
exist = table.exists(get);
assertEquals(true, exist);
Result result = table.get(get);
// this will fail saying that the Result is empty
// if we remove the exist everything is fine
assertEquals(false, result.isEmpty());
assertTrue(Bytes.equals(VALUE, result.getValue(FAMILY, QUALIFIER)));
{code}
if we use a different Get instance for the get everything works
{code}
...
get = new Get(ROW);
Result result = table.get(get);
assertEquals(false, result.isEmpty());
{code}
HTable.exists() set the checkExistenceOnly flag in the Get so that object is
not reusable by a table.get()
{code}
public boolean exists(final Get get) throws IOException {
get.setCheckExistenceOnly(true);
Result r = get(get);
assert r.getExists() != null;
return r.getExists();
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
