[jira] [Updated] (HBASE-5708) [89-fb] Make MiniMapRedCluster directory a subdirectory of target/test
[ https://issues.apache.org/jira/browse/HBASE-5708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5708: --- Attachment: D2601.3.patch mbautin updated the revision [jira] [HBASE-5708] [89-fb] Improving robustness of map-reduce-related and other unit tests. Reviewers: Kannan, Karthik, Liyin, JIRA, khemani A few more improvements: - Fixing a bug in handling of MSG_REGION_OPEN retries in case root region is unavailable. Previously we would put the event into the queue but handle it right away anyway, which resulted in missing znode exceptions down the road. - Fixing how we wait until ZK session expires after we force it to close in HBaseTestingUtility. The polling delay should be constant and should not depend on ZK session timeout. - Increasing TTL in TestScannerSelectionUsingTTL to 10 seconds. With the previous value of two seconds some of the newer HFiles had time to expire by the time they were read under high load conditions (multiple tests running on a machine). - Adding extra delay tolerance to a few more wait loops in TestSplitLogManager. This also came up under high system load. - Increasing the number of client retries for tests from 10 to 100. This came up in some unit test failures. Unfortunately, there are still unit test failures, especially if every test is run 5 times. Those are not easy to fix, but we will gradually isolate and fix them. As it turns out, quite a few bugs could be found by simply running unit tests multiple times. However, this patch does reduce the number of various test problems from 64 to 28 (counting test method failures, test method errors, and timeouts for the whole test class) when every test class is run 5 times. I think this is a good step towards having a stable test suite. REVISION DETAIL https://reviews.facebook.net/D2601 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/HConstants.java src/main/java/org/apache/hadoop/hbase/master/OldLogsCleaner.java src/main/java/org/apache/hadoop/hbase/master/RegionManager.java src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java src/test/java/org/apache/hadoop/hbase/TestFullLogReconstruction.java src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java src/test/java/org/apache/hadoop/hbase/filter/TestColumnPrefixFilter.java src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestFixedFileTrailer.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileDataBlockEncoder.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFilePerformance.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestScannerSelectionUsingTTL.java src/test/java/org/apache/hadoop/hbase/mapred/TestLegacyTableMapReduce.java src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java src/test/java/org/apache/hadoop/hbase/regionserver/TestColumnSeeking.java src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactSelection.java src/test/java/org/apache/hadoop/hbase/regionserver/TestCompoundBloomFilter.java src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java src/test/java/org/apache/hadoop/hbase/regionserver/TestMultiColumnScanner.java src/test/java/org/apache/hadoop/hbase/regionserver/TestScannerResets.java src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogMethods.java src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java
[jira] [Commented] (HBASE-5708) [89-fb] Make MiniMapRedCluster directory a subdirectory of target/test
[ https://issues.apache.org/jira/browse/HBASE-5708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247095#comment-13247095 ] Phabricator commented on HBASE-5708: mbautin has commented on the revision [jira] [HBASE-5708] [89-fb] Improving robustness of map-reduce-related and other unit tests. INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/master/OldLogsCleaner.java:98-100 Saw a NullPointerException on the next line. src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java:984-987 Saw an infinite loop with a SESSIONEXPIRED here in TestSplitLogManager. src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java:1070-1073 Saw an infinite loop with a SESSIONEXPIRED here in TestSplitLogManager. src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java:1591 Previously, we would proceed to process the open region message, even though we have re-queued it for later. src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java:52 A lot of stuff in this deprecated class should eventually be delegated to HBaseTestingUtility. src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java:138-139 Sometimes we just want to reuse the mini-map-reduce-cluster-starting capabilities of HBaseTestingUtility, but provide a NameNode URI ourselves. src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java:364 Observed failures to start namenode's trash cleaning thread when it failed to connect as a client to the newly started namenode. Adding a delay fixed the problem in some of those cases. src/test/java/org/apache/hadoop/hbase/util/TestMiniClusterLoadSequential.java:80 The load balancer would unassign some regions in a load-test unit test, resulting in a failure, which is totally ridiculous. src/test/resources/hbase-site.xml:145 A lot of tests used to fail with ZK session expiration errors. I have set ZK session timeout to 1 sec in a few places where it seemed necessary. src/test/resources/hbase-site.xml:155 We did not have enough retries in some unit tests. This will hopefully add some robustness. REVISION DETAIL https://reviews.facebook.net/D2601 [89-fb] Make MiniMapRedCluster directory a subdirectory of target/test -- Key: HBASE-5708 URL: https://issues.apache.org/jira/browse/HBASE-5708 Project: HBase Issue Type: Bug Reporter: Mikhail Bautin Priority: Minor Attachments: D2601.1.patch, D2601.2.patch, D2601.3.patch Some map-reduce-based tests are failing when executed concurrently in 89-fb because mini-map-reduce cluster uses /tmp/hadoop-username for temporary data. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5644) [findbugs] Fix null pointer warnings.
[ https://issues.apache.org/jira/browse/HBASE-5644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247097#comment-13247097 ] Jonathan Hsieh commented on HBASE-5644: --- First quick review (I have more to review, will continue hopefully in next day) Test failure on TestSplitTransactionOnCluster seems to be legit -- it hung on my machine locally 2x. Have you used Preconditions before? It is slightly more compact and seems little better at conveying intent. http://google-collections.googlecode.com/svn/trunk/javadoc/com/google/common/base/Preconditions.html There was another NP findbugs warnings from build 1365: Mind handling this one too? Possible null pointer dereference of serverInfo in org.apache.hadoop.hbase.tmpl.regionserver.RSStatusTmplImpl.renderNoFlush(Writer) on exception path [findbugs] Fix null pointer warnings. - Key: HBASE-5644 URL: https://issues.apache.org/jira/browse/HBASE-5644 Project: HBase Issue Type: Sub-task Components: scripts Reporter: Jonathan Hsieh Assignee: Uma Maheswara Rao G Attachments: HBASE-5644.patch, NullPointerFindBugs_Analysis.xlsx See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Fix the NP category -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5722) NPE in ZKUtil#getChildDataAndWatchForNewChildren when ZK not available or NW down.
NPE in ZKUtil#getChildDataAndWatchForNewChildren when ZK not available or NW
down.
--
Key: HBASE-5722
URL: https://issues.apache.org/jira/browse/HBASE-5722
Project: HBase
Issue Type: Bug
Components: zookeeper
Affects Versions: 0.96.0
Reporter: Uma Maheswara Rao G
Assignee: Uma Maheswara Rao G
{code}
ListString nodes =
ZKUtil.listChildrenAndWatchForNewChildren(zkw, baseNode);
ListNodeAndData newNodes = new ArrayListNodeAndData();
for (String node: nodes) {
String nodePath = ZKUtil.joinZNode(baseNode, node);
byte [] data = ZKUtil.getDataAndWatch(zkw, nodePath);
newNodes.add(new NodeAndData(nodePath, data));
}
{code}
The above code can throw NPE when listChildrenAndWatchForNewChildren returns
null.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3909) Add dynamic config
[
https://issues.apache.org/jira/browse/HBASE-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247113#comment-13247113
]
Uma Maheswara Rao G commented on HBASE-3909:
@Subbu,
I just reviewed this patch. Feedback as follows.
| 1) updateClusterConfig and createClusterConfig are looks like mostly
duplicate code for me. Can we reuse the code or can change the API, such that
it can handle both.
{code}
if (ZKUtil.checkExists(watcher,
+getConfigNodePath(configKey)) 0) {
+LOG.info(Cluster configuration node exist for Config Key =
++ configKey + Updating the cluster config node);
+ZKUtil.setData(watcher, getConfigNodePath(configKey),
Bytes.toBytes(configValue));
+} else {
+ZKUtil.createSetData(watcher, getConfigNodePath(configKey),
+Bytes.toBytes(configValue));
+}
{code}
| 2) Looks your are using wrong code formatter. intendation should be 2 spaces.
{code}
try {
+if (ZKUtil.checkExists(watcher,
{code}
| 3) Normally we used to put the licence header at top of the file.
{code}
+package org.apache.hadoop.hbase.zookeeper;
+
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.Abortable;
+import org.apache.hadoop.hbase.HBaseInMemoryConfiguration;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.zookeeper.KeeperException;
+import org.codehaus.jackson.map.ObjectMapper;
+
+import java.io.IOException;
+import java.io.StringWriter;
+import java.util.Map;
+
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
{code}
| 4) configuration.dumpConfiguration(configuration, outWriter); , this api is
static in configuration , you can call directly with class.
| 5) getMaster in HBaseAdmin seems like deprecated
{code}
return getMaster().updateConfig(configKey, configValue);
{code}
| 6) please add the javadoc cleanly
{code}
/**
+ *
+ * @param configKey
+ * @param configValue
+ * @return
+ */
{code}
| 7) a) Call back comes from ZK for dataChangeEvent,
b) when you are processing the event at Hmaster/Regionserver to refresh the
configurations.
c) Now unfortunately ZK down/nw fluctuation from ZK to Server.
d) It may get data as null. And refreshClusterConfigMap will just ignore the
dataupdation in memory.
Then this node may continue with old configs right? when this will get
updated again?
seems to be a bug here. no?
You have used ZKUtil#getChildDataAndWatchForNewChildren, it can throw NPE, I
Just filed JIRA HBASE-5722
| 8) Looks you are creating znodes for all the configuration items.
I feel, we should not allow user to change all the configuration items. (
like some configurations we may not be able to change at runtime, may be zk url
etc..).
So, while creating znodes it self we can create only for mutable config
items in ZK. If user tries to update any immutable config items, you can
directly emit the warning.
Am i missing some thing here?
| 9) I think we may need to add the validation checks for the mutable
configurations from admin. Since we are encouraging the users to update configs
dynamically, if user sets some wrong values, it may change directly and system
may go immediatly to unstable state. may be dangerous. no?
| 10) Looks we are handling nodeDeleted event. When exactly this event can come?
I think we added updateConfig api in HBase admin. How can we remove
element with this? Is this event really will occur?
| 11) Final question is that, we are claiming hadoop can run in commodity
hardwares, in that case some configuration items can be different in each
machine. for example: 'hbase.regionserver.handler.count'..etc
| 12)createClusterConfig creating JSonConfiguration from the Configuration and
adding them into HBaseInMemoryConfiguration.
Can't we create update the HBaseInMemoryConfiguration from input
Configuration.
Please correct me if I missunderstand your design. Great work thanks a lot.
Thanks
Uma
Add dynamic config
--
Key: HBASE-3909
URL: https://issues.apache.org/jira/browse/HBASE-3909
Project: HBase
Issue Type: Bug
Reporter: stack
Fix For: 0.96.0
Attachments: 3909-v1.patch, 3909.v1
I'm sure this issue exists already, at least as part of the discussion around
making online schema edits possible, but no hard this having its own issue.
Ted started a conversation on this topic up on dev and Todd suggested we
lookd at how Hadoop did it over in HADOOP-7001
--
This message is
[jira] [Commented] (HBASE-3909) Add dynamic config
[
https://issues.apache.org/jira/browse/HBASE-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247116#comment-13247116
]
Uma Maheswara Rao G commented on HBASE-3909:
Point 11) is incomplete:
{quote}
11) Final question is that, we are claiming hadoop can run in commodity
hardwares, in that case some configuration items can be different in each
machine. for example: 'hbase.regionserver.handler.count'..etc
{quote}
But the zookeeper based design may update the configurations in all nodes as
same.
We can't provide dynamic configuration updation functionality for such kind of
properties right? If you see one property in DataNode, dataXceiverCount. We can
configure this property defferently in each machine, and also this is a
dynamically changeable item.
Add dynamic config
--
Key: HBASE-3909
URL: https://issues.apache.org/jira/browse/HBASE-3909
Project: HBase
Issue Type: Bug
Reporter: stack
Fix For: 0.96.0
Attachments: 3909-v1.patch, 3909.v1
I'm sure this issue exists already, at least as part of the discussion around
making online schema edits possible, but no hard this having its own issue.
Ted started a conversation on this topic up on dev and Todd suggested we
lookd at how Hadoop did it over in HADOOP-7001
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5723) Simple Design of Secondary Index
Simple Design of Secondary Index Key: HBASE-5723 URL: https://issues.apache.org/jira/browse/HBASE-5723 Project: HBase Issue Type: New Feature Components: coprocessors Reporter: ShiXing Priority: Minor Attachments: Simple Design of HBase SecondaryIndex.pdf Use coprocessor to create index. And primary tables' compaction to purge the stale data. Attach file is the Design of the Seconday Index. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5723) Simple Design of Secondary Index
[ https://issues.apache.org/jira/browse/HBASE-5723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ShiXing updated HBASE-5723: --- Attachment: Simple Design of HBase SecondaryIndex.pdf Simple Design of Secondary Index Key: HBASE-5723 URL: https://issues.apache.org/jira/browse/HBASE-5723 Project: HBase Issue Type: New Feature Components: coprocessors Reporter: ShiXing Priority: Minor Attachments: Simple Design of HBase SecondaryIndex.pdf Use coprocessor to create index. And primary tables' compaction to purge the stale data. Attach file is the Design of the Seconday Index. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5723) Simple Design of Secondary Index
[ https://issues.apache.org/jira/browse/HBASE-5723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247127#comment-13247127 ] ShiXing commented on HBASE-5723: Difficulty and risk 1. During primary table compact, the StoreScanner just read family.getMaxVersions() versions' data, the old data will be skiped, but the old data's indices should be deleted, So we must read all version data for indices' delete, and family.getMaxVersions() version for store's compaction. Also the timestamp of columns that we want to delete the indices should also have not TimeRange, now it is System.CurrentTimeMillis() - ttl; Simple Design of Secondary Index Key: HBASE-5723 URL: https://issues.apache.org/jira/browse/HBASE-5723 Project: HBase Issue Type: New Feature Components: coprocessors Reporter: ShiXing Priority: Minor Attachments: Simple Design of HBase SecondaryIndex.pdf Use coprocessor to create index. And primary tables' compaction to purge the stale data. Attach file is the Design of the Seconday Index. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5724) Row cache of KeyValue should be cleared in readFields().
Row cache of KeyValue should be cleared in readFields(). Key: HBASE-5724 URL: https://issues.apache.org/jira/browse/HBASE-5724 Project: HBase Issue Type: Bug Affects Versions: 0.92.1 Reporter: Teruyoshi Zenmyo KeyValue does not clear its row cache in reading new values (readFileds()). Therefore, If a KeyValue (kv) which caches its row bytes reads another KeyValue instance, kv.getRow() returns a wrong value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5724) Row cache of KeyValue should be cleared in readFields().
[ https://issues.apache.org/jira/browse/HBASE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teruyoshi Zenmyo updated HBASE-5724: Attachment: HBASE-5724.txt The attached file is a patch which adds just one line together with a new test. - set rowcache of KeyValue to null in the readFields() - one new test which makes sure KeyValue.getRow() returns a correct value. Row cache of KeyValue should be cleared in readFields(). Key: HBASE-5724 URL: https://issues.apache.org/jira/browse/HBASE-5724 Project: HBase Issue Type: Bug Affects Versions: 0.92.1 Reporter: Teruyoshi Zenmyo Attachments: HBASE-5724.txt KeyValue does not clear its row cache in reading new values (readFileds()). Therefore, If a KeyValue (kv) which caches its row bytes reads another KeyValue instance, kv.getRow() returns a wrong value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5724) Row cache of KeyValue should be cleared in readFields().
[ https://issues.apache.org/jira/browse/HBASE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teruyoshi Zenmyo updated HBASE-5724: Status: Patch Available (was: Open) Row cache of KeyValue should be cleared in readFields(). Key: HBASE-5724 URL: https://issues.apache.org/jira/browse/HBASE-5724 Project: HBase Issue Type: Bug Affects Versions: 0.92.1 Reporter: Teruyoshi Zenmyo Attachments: HBASE-5724.txt KeyValue does not clear its row cache in reading new values (readFileds()). Therefore, If a KeyValue (kv) which caches its row bytes reads another KeyValue instance, kv.getRow() returns a wrong value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1936) ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services
[ https://issues.apache.org/jira/browse/HBASE-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247180#comment-13247180 ] Jieshan Bean commented on HBASE-1936: - @stack, can you look back to this new feature again? I think it's very useful either for hot deployment of Filter or dynamic coprocessor. Any potential reasons for hanging this task up? Thank you. ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services --- Key: HBASE-1936 URL: https://issues.apache.org/jira/browse/HBASE-1936 Project: HBase Issue Type: New Feature Reporter: stack Assignee: Daniel Ploeg Labels: noob Attachments: cp_from_hdfs.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5599) The hbck tool can not fix the six scenarios, it is NO_VERSION_FILE, NOT_IN_META_OR_DEPLOYED, NOT_IN_META, SHOULD_NOT_BE_DEPLOYED, FIRST_REGION_STARTKEY_NOT_EMPTY, HOLE_
[ https://issues.apache.org/jira/browse/HBASE-5599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247197#comment-13247197 ] fulin wang commented on HBASE-5599: --- The SHOULD_NOT_BE_DEPLOYED fault scenario, I am thinking about how to write this test case. Because the fault scenario appear when the table is disabled, but the region of this table is not closed. The hbck tool can not fix the six scenarios, it is NO_VERSION_FILE, NOT_IN_META_OR_DEPLOYED, NOT_IN_META, SHOULD_NOT_BE_DEPLOYED, FIRST_REGION_STARTKEY_NOT_EMPTY, HOLE_IN_REGION_CHAIN. Key: HBASE-5599 URL: https://issues.apache.org/jira/browse/HBASE-5599 Project: HBase Issue Type: New Feature Components: hbck Affects Versions: 0.90.6 Reporter: fulin wang Assignee: fulin wang Fix For: 0.90.6 Attachments: hbase-5599-0.90.patch, hbase-5599-0.90_v2.patch, hbase-5599-0.90_v3.patch, hbase-5599-0.90_v5.patch, hbase-5599-0.90_v6.patch, hbase-5599-0.92_v5.patch, hbase-5599-0.94_v5.patch, hbase-5599-trunk_v5.patch The hbck tool can not fix the six scenarios. 1. Version file does not exist in root dir. Fix: I try to create a version file by 'FSUtils.setVersion' method. 2. [REGIONNAME][KEY] on HDFS, but not listed in META or deployed on any region server. Fix: I get region info form the hdfs file, this region info write to '.META.' table. 3. [REGIONNAME][KEY] not in META, but deployed on [SERVERNAME] Fix: I get region info form the hdfs file, this region info write to '.META.' table. 4. [REGIONNAME] should not be deployed according to META, but is deployed on [SERVERNAME] Fix: Close this region. 5. First region should start with an empty key. You need to create a new region and regioninfo in HDFS to plug the hole. Fix: The region info is not in hdfs and .META., so it create a empty region for this error. 6. There is a hole in the region chain between [KEY] and [KEY]. You need to create a new regioninfo and region dir in hdfs to plug the hole. Fix: The region info is not in hdfs and .META., so it create a empty region for this hole. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5725) NPE in
NPE in --- Key: HBASE-5725 URL: https://issues.apache.org/jira/browse/HBASE-5725 Project: HBase Issue Type: Bug Reporter: Uma Maheswara Rao G When i am running TestSplitTransactionOnCluster, it is throwing NPE from HBaseClient HBaseClient: RpcResponse response = RpcResponse.parseDelimitedFrom(in); int id = response.getCallId(); The above code throws NPE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5725) NPE in
[
https://issues.apache.org/jira/browse/HBASE-5725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247201#comment-13247201
]
Uma Maheswara Rao G commented on HBASE-5725:
More info:
{quote}
2012-04-05 18:35:14,860 WARN [IPC Client (47) connection to
umamahesh.com/xx.xx.xx.127:1109 from U72686.hfs.4]
ipc.HBaseClient$Connection(510): Unexpected exception receiving call responses
java.lang.NullPointerException
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:566)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:507)
2012-04-05 18:35:14,860 WARN
[RegionServer:4;umamahesh.china.huawei.com,1478,1333631106094-splits-1333631113219]
client.HConnectionManager$HConnectionImplementation(1894): Failed all from
region=.META.,,1.1028785192, hostname=umamahesh.china.huawei.com, port=1109
java.util.concurrent.ExecutionException: java.io.IOException: Call to
umamahesh.com/xx.xx.xx.127:1109 failed on local exception: java.io.IOException:
Unexpected exception receiving call responses
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1864)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1715)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:955)
at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:811)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:786)
at org.apache.hadoop.hbase.catalog.MetaEditor.put(MetaEditor.java:100)
at
org.apache.hadoop.hbase.catalog.MetaEditor.putToMetaTable(MetaEditor.java:67)
at
org.apache.hadoop.hbase.catalog.MetaEditor.offlineParentInMeta(MetaEditor.java:189)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.createDaughters(SplitTransaction.java:319)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.execute(SplitTransaction.java:452)
at
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:69)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
{quote}
NPE in
---
Key: HBASE-5725
URL: https://issues.apache.org/jira/browse/HBASE-5725
Project: HBase
Issue Type: Bug
Reporter: Uma Maheswara Rao G
When i am running TestSplitTransactionOnCluster, it is throwing NPE from
HBaseClient
HBaseClient:
RpcResponse response = RpcResponse.parseDelimitedFrom(in);
int id = response.getCallId();
The above code throws NPE.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5725) HBaseClient throws NPE while in Connection#receiveResponse call.
[ https://issues.apache.org/jira/browse/HBASE-5725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HBASE-5725: --- Summary: HBaseClient throws NPE while in Connection#receiveResponse call. (was: NPE in ) HBaseClient throws NPE while in Connection#receiveResponse call. Key: HBASE-5725 URL: https://issues.apache.org/jira/browse/HBASE-5725 Project: HBase Issue Type: Bug Reporter: Uma Maheswara Rao G When i am running TestSplitTransactionOnCluster, it is throwing NPE from HBaseClient HBaseClient: RpcResponse response = RpcResponse.parseDelimitedFrom(in); int id = response.getCallId(); The above code throws NPE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5724) Row cache of KeyValue should be cleared in readFields().
[ https://issues.apache.org/jira/browse/HBASE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teruyoshi Zenmyo updated HBASE-5724: Description: KeyValue does not clear its row cache in reading new values (readFields()). Therefore, If a KeyValue (kv) which caches its row bytes reads another KeyValue instance, kv.getRow() returns a wrong value. was: KeyValue does not clear its row cache in reading new values (readFileds()). Therefore, If a KeyValue (kv) which caches its row bytes reads another KeyValue instance, kv.getRow() returns a wrong value. Row cache of KeyValue should be cleared in readFields(). Key: HBASE-5724 URL: https://issues.apache.org/jira/browse/HBASE-5724 Project: HBase Issue Type: Bug Affects Versions: 0.92.1 Reporter: Teruyoshi Zenmyo Attachments: HBASE-5724.txt KeyValue does not clear its row cache in reading new values (readFields()). Therefore, If a KeyValue (kv) which caches its row bytes reads another KeyValue instance, kv.getRow() returns a wrong value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5726) TestSplitTransactionOnCluster occasionally failing
TestSplitTransactionOnCluster occasionally failing
--
Key: HBASE-5726
URL: https://issues.apache.org/jira/browse/HBASE-5726
Project: HBase
Issue Type: Bug
Reporter: Uma Maheswara Rao G
When I ran TestSplitTransactionOnCluster, some times tests are failing.
{quote}
java.lang.AssertionError: expected:1 but was:0
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.getAndCheckSingleTableRegion(TestSplitTransactionOnCluster.java:89)
at
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testShutdownFixupWhenDaughterHasSplit(TestSplitTransactionOnCluster.java:298)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
{quote}
Seems like test is flaky, random other cases also fails.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5724) Row cache of KeyValue should be cleared in readFields().
[ https://issues.apache.org/jira/browse/HBASE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247244#comment-13247244 ] Hadoop QA commented on HBASE-5724: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12521490/HBASE-5724.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestFromClientSide org.apache.hadoop.hbase.mapreduce.TestMultithreadedTableMapper org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat org.apache.hadoop.hbase.mapreduce.TestTableMapReduce Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1403//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1403//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1403//console This message is automatically generated. Row cache of KeyValue should be cleared in readFields(). Key: HBASE-5724 URL: https://issues.apache.org/jira/browse/HBASE-5724 Project: HBase Issue Type: Bug Affects Versions: 0.92.1 Reporter: Teruyoshi Zenmyo Attachments: HBASE-5724.txt KeyValue does not clear its row cache in reading new values (readFields()). Therefore, If a KeyValue (kv) which caches its row bytes reads another KeyValue instance, kv.getRow() returns a wrong value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4818) HBase Shell - Add support for formatting row keys before output
[
https://issues.apache.org/jira/browse/HBASE-4818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247246#comment-13247246
]
Ben West commented on HBASE-4818:
-
I can work on adding this to the web UI if someone can suggest a place to store
the formatter preference.
Should it just be in hbase-site.xml?
HBase Shell - Add support for formatting row keys before output
---
Key: HBASE-4818
URL: https://issues.apache.org/jira/browse/HBASE-4818
Project: HBase
Issue Type: Improvement
Components: shell
Reporter: Eran Kampf
Priority: Trivial
Attachments: format3.patch, hbase-4818.patch
Original Estimate: 24h
Remaining Estimate: 24h
As many HBase users use binary row keys rather than strings to optimize
memory consumption displaying an escaped string in the HBase shell isn't
useful (and takes a lot of screen space)
Allowing user to provide a row key formatter as part of the scan\get commands
would allow developers to display the row key in a way thats makes sense for
them.
Example:
scan 'stats', { ROWFORMATTER = MyRowFormatter.new }
The row formatter simply gets the bytes array key and formats it to a string.
Its an easy change tomake with simple monkey-patching of the shell commands
but I would be happy to see it as part of the shell itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5644) [findbugs] Fix null pointer warnings.
[
https://issues.apache.org/jira/browse/HBASE-5644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247253#comment-13247253
]
Uma Maheswara Rao G commented on HBASE-5644:
Thanks a lot Jon for taking a look.
{quote}
Test failure on TestSplitTransactionOnCluster seems to be legit – it hung on my
machine locally 2x.
{quote}
Looks TestSplitTransactionOnCluster is failing randomly with out the patch as
well. Just filed HBASE-5726.
{quote}
Have you used Preconditions before? It is slightly more compact and seems
little better at conveying intent.
{quote}
Used, but not extensively ( only in Hadoop). are you suggesting not to use it
here?
{quote}
There was another NP findbugs warnings from build 1365: Mind handling this one
too?
Possible null pointer dereference of serverInfo in
org.apache.hadoop.hbase.tmpl.regionserver.RSStatusTmplImpl.renderNoFlush(Writer)
on exception path
{quote}
Yes, I added this comment remarks in my analysis sheet.
RSStatusTmplImpl is Autogenerated Jamon code. I wanted to fix it but I am not
familiar in Jamon code generation area :(. Let some one or you can update this
change if you are familiar. If not, can I file a separate bug?
{code}
HServerInfo serverInfo = null;
ServerName serverName = null;
try {
serverInfo = regionServer.getHServerInfo();
serverName = regionServer.getServerName();
} catch (IOException e) {
e.printStackTrace();
}
{code}
It is ignoring the exception here and proceeding and using serverInfo. This can
cause NPE here if getHServerInfo throws IOE.
[findbugs] Fix null pointer warnings.
-
Key: HBASE-5644
URL: https://issues.apache.org/jira/browse/HBASE-5644
Project: HBase
Issue Type: Sub-task
Components: scripts
Reporter: Jonathan Hsieh
Assignee: Uma Maheswara Rao G
Attachments: HBASE-5644.patch, NullPointerFindBugs_Analysis.xlsx
See
https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Fix the NP category
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5721) Update bundled hadoop to be 1.0.2 (it was just released)
[ https://issues.apache.org/jira/browse/HBASE-5721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247260#comment-13247260 ] Zhihong Yu commented on HBASE-5721: --- Seems to be the same as HBASE-5471 Update bundled hadoop to be 1.0.2 (it was just released) Key: HBASE-5721 URL: https://issues.apache.org/jira/browse/HBASE-5721 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Attachments: 1.0.2.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5721) Update bundled hadoop to be 1.0.2 (it was just released)
[ https://issues.apache.org/jira/browse/HBASE-5721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5721: -- Attachment: 5721.txt Update bundled hadoop to be 1.0.2 (it was just released) Key: HBASE-5721 URL: https://issues.apache.org/jira/browse/HBASE-5721 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Attachments: 1.0.2.txt, 5721.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5724) Row cache of KeyValue should be cleared in readFields().
[
https://issues.apache.org/jira/browse/HBASE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247267#comment-13247267
]
Zhihong Yu commented on HBASE-5724:
---
How about making the following comment in test clearer ?
{code}
+ * make sure a row cache is cleared after a new value is read.
{code}
The row cache is cleared and re-read for the new value.
Row cache of KeyValue should be cleared in readFields().
Key: HBASE-5724
URL: https://issues.apache.org/jira/browse/HBASE-5724
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.1
Reporter: Teruyoshi Zenmyo
Attachments: HBASE-5724.txt
KeyValue does not clear its row cache in reading new values (readFields()).
Therefore, If a KeyValue (kv) which caches its row bytes reads another
KeyValue instance, kv.getRow() returns a wrong value.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5471) Upgrade hadoop to 1.0.2
[ https://issues.apache.org/jira/browse/HBASE-5471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-5471. -- Resolution: Duplicate Resolving as duplicate of hbase-5721 (though this was created first -- I should have used this one) Upgrade hadoop to 1.0.2 --- Key: HBASE-5471 URL: https://issues.apache.org/jira/browse/HBASE-5471 Project: HBase Issue Type: Task Reporter: Zhihong Yu Now that MAPREDUCE-3583 has been integrated, we should be prepared to upgrade hadoop to 1.0.2 Unit tests which fail on Apache QA machines due to NumberFormatException should pass after upgrade. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5721) Update bundled hadoop to be 1.0.2 (it was just released)
[ https://issues.apache.org/jira/browse/HBASE-5721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5721: - Status: Open (was: Patch Available) Update bundled hadoop to be 1.0.2 (it was just released) Key: HBASE-5721 URL: https://issues.apache.org/jira/browse/HBASE-5721 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Attachments: 1.0.2.txt, 5721.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5721) Update bundled hadoop to be 1.0.2 (it was just released)
[ https://issues.apache.org/jira/browse/HBASE-5721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5721: - Status: Patch Available (was: Open) Update bundled hadoop to be 1.0.2 (it was just released) Key: HBASE-5721 URL: https://issues.apache.org/jira/browse/HBASE-5721 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Attachments: 1.0.2.txt, 5721.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5724) Row cache of KeyValue should be cleared in readFields().
[ https://issues.apache.org/jira/browse/HBASE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247291#comment-13247291 ] stack commented on HBASE-5724: -- +1 on patch. Row cache of KeyValue should be cleared in readFields(). Key: HBASE-5724 URL: https://issues.apache.org/jira/browse/HBASE-5724 Project: HBase Issue Type: Bug Affects Versions: 0.92.1 Reporter: Teruyoshi Zenmyo Attachments: HBASE-5724.txt KeyValue does not clear its row cache in reading new values (readFields()). Therefore, If a KeyValue (kv) which caches its row bytes reads another KeyValue instance, kv.getRow() returns a wrong value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5727) secure hbase build broke because of 'HBASE-5451 Switch RPC call envelope/headers to PBs'
secure hbase build broke because of 'HBASE-5451 Switch RPC call envelope/headers to PBs' Key: HBASE-5727 URL: https://issues.apache.org/jira/browse/HBASE-5727 Project: HBase Issue Type: Bug Reporter: stack Assignee: Devaraj Das Priority: Blocker If you build with the security profile -- i.e. add '-P security' on the command line -- you'll see that the secure build is broke since we messed in rpc. Assigning Deveraj to take a look. If you can't work on this now DD, just give it back to me and I'll have a go at it. Thanks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5721) Update bundled hadoop to be 1.0.2 (it was just released)
[ https://issues.apache.org/jira/browse/HBASE-5721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247299#comment-13247299 ] Hadoop QA commented on HBASE-5721: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12521503/5721.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1404//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1404//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1404//console This message is automatically generated. Update bundled hadoop to be 1.0.2 (it was just released) Key: HBASE-5721 URL: https://issues.apache.org/jira/browse/HBASE-5721 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Attachments: 1.0.2.txt, 5721.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5615) the master never does balance because of balancing the parent region
[ https://issues.apache.org/jira/browse/HBASE-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan reopened HBASE-5615: --- This issue fix is not valid for 0.92 and above branches as it uses ZK for splitting. the master never does balance because of balancing the parent region Key: HBASE-5615 URL: https://issues.apache.org/jira/browse/HBASE-5615 Project: HBase Issue Type: Bug Affects Versions: 0.90.7 Reporter: xufeng Assignee: xufeng Priority: Critical Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: 5615-trunk.txt, HBASE-5615-90.patch, HBASE-5615.patch, NoPatched-surefire-report-5615-90.html, Patched_surefire-report-5615-90.html the master never do balance becauseof when master do rebuildUserRegions()，it will add the parent region into AssignmentManager#servers, if balancer let the parent region to move,the parent will in RIT forever.thus balance will never be executed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5724) Row cache of KeyValue should be cleared in readFields().
[
https://issues.apache.org/jira/browse/HBASE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5724:
--
Comment: was deleted
(was: How about making the following comment in test clearer ?
{code}
+ * make sure a row cache is cleared after a new value is read.
{code}
The row cache is cleared and re-read for the new value.)
Row cache of KeyValue should be cleared in readFields().
Key: HBASE-5724
URL: https://issues.apache.org/jira/browse/HBASE-5724
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.1
Reporter: Teruyoshi Zenmyo
Attachments: HBASE-5724.txt
KeyValue does not clear its row cache in reading new values (readFields()).
Therefore, If a KeyValue (kv) which caches its row bytes reads another
KeyValue instance, kv.getRow() returns a wrong value.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5504) Online Merge
[
https://issues.apache.org/jira/browse/HBASE-5504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247307#comment-13247307
]
stack commented on HBASE-5504:
--
Hey Eric. Thanks for the help. Looking at FATE it didn't look like we could
pull it over easily (maybe I should look again) but for sure we need to emulate
something like it. First up would be table read/write lock and have actions
like split (or bulk import) take out at a read lock before progressing. Can I
have a pointer for your file GCer, on why delayed cleanup? So you'd keep up in
meta the list of files for a range and this would be the region/tablets' list
even though the files might not sit physically under a particular region/tablet
(What about compactions? Would it have to update the list in the meta table
when it moved the compacted file into place?). On point 1., isn't it possible
a file might still over-span a range?
Good stuff.
Online Merge
Key: HBASE-5504
URL: https://issues.apache.org/jira/browse/HBASE-5504
Project: HBase
Issue Type: Brainstorming
Components: client, master, shell, zookeeper
Affects Versions: 0.94.0
Reporter: Mubarak Seyed
Fix For: 0.96.0
As discussed, please refer the discussion at
[HBASE-4991|https://issues.apache.org/jira/browse/HBASE-4991]
Design suggestion from Stack:
{quote}
I suggest a design below. It has some prerequisites, some general function
that this feature could use (and others). The prereqs if you think them good,
could be done outside of this JIRA.
Here's a suggested rough outline of how I think this feature should run. The
feature I'm describing below is merge and deleteRegion for I see them as in
essence the same thing.
(C) Client, (M) Master, RS (Region server), ZK (ZooKeeper)
1. Client calls merge or deleteRegion API. API is a range of rows. (C)
2. Master gets call. (M)
3. Master obtains a write lock on table so it can't be disabled from under
us. The write lock will also disable splitting. This is one of the prereqs I
think. Its HBASE-5494 (Or we could just do something simpler where we have a
flag up in zk that splitRegion checks but thats less useful I think; OR we do
the dynamic configs issue and set splits to off via a config. change).
There'd be a timer for how long we wait on the table lock. (M - ZK)
4. If we get the lock, write intent to merge a range up into zk. It also
hoists into zk if its a pure merge or a merge that drops the region data (a
deleteRegion call) (M)
5. Return to the client either our failed attempt at locking the table or an
id of some sort used identifying this running operation; can use it querying
status. (M - C)
6. Turn off balancer. TODO/prereq: Do it in a way that is persisted. Balancer
switch currently in memory only so if master crashes, new master will come up
in balancing mode # (If we had dynamic config. could hoist up to zk a config.
that disables the balancer rather than have a balancer-specific flag/znode OR
if a write lock outstanding on a table, then the balancer does not balance
regions in the locked table - this latter might be the easiest to do) (M)
7. Write into zk that just turned off the balancer (If it was on) (M - ZK)
8. Get regions that are involved in the span (M)
9. Hoist the list up into zk. (M - ZK)
10. Create region to span the range. (M)
11. Write that we did this up into zk. (M - ZK)
12. Close regions in parallel. Confirm close in parallel. (M - RS)
13. Write up into zk regions closed (This might not be necessary since can
ask if region is open). (M - ZK)
14. If a merge and not a delete region, move files under new region. Might
multithread this (moves should go pretty fast). If a deleteregion, we skip
this step. (M)
15. On completion mark zk (though may not be necessary since its easy to look
in fs to see state of move). (M - ZK)
16. Edit .META. (M)
17. Confirm edits went in. (M)
18. Move old regions to hbase trash folder TODO: There is no trash folder
under /hbase currently. We should add one. (M)
19. Enable balancer (if it was off) (M)
20. Unlock table (M)
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1936) ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services
[ https://issues.apache.org/jira/browse/HBASE-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247309#comment-13247309 ] ramkrishna.s.vasudevan commented on HBASE-1936: --- @Jieshan Thanks for bringing this up. ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services --- Key: HBASE-1936 URL: https://issues.apache.org/jira/browse/HBASE-1936 Project: HBase Issue Type: New Feature Reporter: stack Assignee: Daniel Ploeg Labels: noob Attachments: cp_from_hdfs.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5728) Methods Missing in HTableInterface
Methods Missing in HTableInterface -- Key: HBASE-5728 URL: https://issues.apache.org/jira/browse/HBASE-5728 Project: HBase Issue Type: Improvement Components: client Reporter: Bing Li Dear all, I found some methods existed in HTable were not in HTableInterface. setAutoFlush setWriteBufferSize ... In most cases, I manipulate HBase through HTableInterface from HTablePool. If I need to use the above methods, how to do that? I am considering writing my own table pool if no proper ways. Is it fine? Thanks so much! Best regards, Bing -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5644) [findbugs] Fix null pointer warnings.
[
https://issues.apache.org/jira/browse/HBASE-5644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247318#comment-13247318
]
Jonathan Hsieh commented on HBASE-5644:
---
Code portion looks good to me.
On the spread sheet, one suggestion:
HTable - row 2, 3
- delete -- why is the type bool? Maybre change to ServerCallableVoid? (two
cases).
Ex:
{code}
@Override
public void delete(final Delete delete)
throws IOException {
new ServerCallableVoid(connection, tableName, delete.getRow(),
operationTimeout) {
public Void call() throws IOException {
server.delete(location.getRegionInfo().getRegionName(), delete);
return null; // FindBugs NP_BOOLEAN_RETURN_NULL
}
}.withRetries();
}
{code}
4,5,6, 7 smells funny but I buy it.
Seems like findbugs doesn't handle ?: very well.
bq. Used, but not extensively ( only in Hadoop). are you suggesting not to use
it here?
What I wrote above was unclear. You use Preconditions in some places (Store),
and there are places you don't (ShutdownHook). Seems like you could us it in a
few more places? Not a big deal, but it makes code easier to read by conveying
more intent IMO.
Maybe you chose not to use because it wasn't at the top of a method?
bq. RSStatusTmplImpl is Autogenerated Jamon code. I wanted to fix it but I am
not familiar in Jamon code generation area . Let some one or you can update
this change if you are familiar. If not, can I file a separate bug?
You can get this one, it is pretty straightforward. The source of the autogen
RSStatusTmplImpl data is here. Take a look, just modify there and it will just
percolate that code through to the java version.
https://github.com/apache/hbase/blob/trunk/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RSStatusTmpl.jamon#L48
[findbugs] Fix null pointer warnings.
-
Key: HBASE-5644
URL: https://issues.apache.org/jira/browse/HBASE-5644
Project: HBase
Issue Type: Sub-task
Components: scripts
Reporter: Jonathan Hsieh
Assignee: Uma Maheswara Rao G
Attachments: HBASE-5644.patch, NullPointerFindBugs_Analysis.xlsx
See
https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Fix the NP category
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5721) Update bundled hadoop to be 1.0.2 (it was just released)
[ https://issues.apache.org/jira/browse/HBASE-5721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5721: - Attachment: 5721.txt Retry Update bundled hadoop to be 1.0.2 (it was just released) Key: HBASE-5721 URL: https://issues.apache.org/jira/browse/HBASE-5721 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Attachments: 1.0.2.txt, 5721.txt, 5721.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5721) Update bundled hadoop to be 1.0.2 (it was just released)
[ https://issues.apache.org/jira/browse/HBASE-5721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5721: - Status: Patch Available (was: Open) Update bundled hadoop to be 1.0.2 (it was just released) Key: HBASE-5721 URL: https://issues.apache.org/jira/browse/HBASE-5721 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Attachments: 1.0.2.txt, 5721.txt, 5721.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5721) Update bundled hadoop to be 1.0.2 (it was just released)
[ https://issues.apache.org/jira/browse/HBASE-5721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5721: - Status: Open (was: Patch Available) Update bundled hadoop to be 1.0.2 (it was just released) Key: HBASE-5721 URL: https://issues.apache.org/jira/browse/HBASE-5721 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Attachments: 1.0.2.txt, 5721.txt, 5721.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5724) Row cache of KeyValue should be cleared in readFields().
[ https://issues.apache.org/jira/browse/HBASE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reassigned HBASE-5724: - Assignee: Teruyoshi Zenmyo Row cache of KeyValue should be cleared in readFields(). Key: HBASE-5724 URL: https://issues.apache.org/jira/browse/HBASE-5724 Project: HBase Issue Type: Bug Affects Versions: 0.92.1 Reporter: Teruyoshi Zenmyo Assignee: Teruyoshi Zenmyo Attachments: HBASE-5724.txt KeyValue does not clear its row cache in reading new values (readFields()). Therefore, If a KeyValue (kv) which caches its row bytes reads another KeyValue instance, kv.getRow() returns a wrong value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5721) Update bundled hadoop to be 1.0.2 (it was just released)
[ https://issues.apache.org/jira/browse/HBASE-5721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5721: - Fix Version/s: 0.94.0 Let's do it for 0.94 as well. Update bundled hadoop to be 1.0.2 (it was just released) Key: HBASE-5721 URL: https://issues.apache.org/jira/browse/HBASE-5721 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Fix For: 0.94.0 Attachments: 1.0.2.txt, 5721.txt, 5721.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5724) Row cache of KeyValue should be cleared in readFields().
[ https://issues.apache.org/jira/browse/HBASE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5724: -- Fix Version/s: 0.96.0 0.94.0 Hadoop Flags: Reviewed Row cache of KeyValue should be cleared in readFields(). Key: HBASE-5724 URL: https://issues.apache.org/jira/browse/HBASE-5724 Project: HBase Issue Type: Bug Affects Versions: 0.92.1 Reporter: Teruyoshi Zenmyo Assignee: Teruyoshi Zenmyo Fix For: 0.94.0, 0.96.0 Attachments: HBASE-5724.txt KeyValue does not clear its row cache in reading new values (readFields()). Therefore, If a KeyValue (kv) which caches its row bytes reads another KeyValue instance, kv.getRow() returns a wrong value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5615) the master never does balance because of balancing the parent region
[ https://issues.apache.org/jira/browse/HBASE-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247325#comment-13247325 ] ramkrishna.s.vasudevan commented on HBASE-5615: --- @Xufeng bq.@Rama I will check the TRUNK and 0.92 version. Did u check for the trunk and 0.92? the master never does balance because of balancing the parent region Key: HBASE-5615 URL: https://issues.apache.org/jira/browse/HBASE-5615 Project: HBase Issue Type: Bug Affects Versions: 0.90.7 Reporter: xufeng Assignee: xufeng Priority: Critical Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: 5615-trunk.txt, HBASE-5615-90.patch, HBASE-5615.patch, NoPatched-surefire-report-5615-90.html, Patched_surefire-report-5615-90.html the master never do balance becauseof when master do rebuildUserRegions()，it will add the parent region into AssignmentManager#servers, if balancer let the parent region to move,the parent will in RIT forever.thus balance will never be executed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-5615) the master never does balance because of balancing the parent region
[ https://issues.apache.org/jira/browse/HBASE-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247325#comment-13247325 ] ramkrishna.s.vasudevan edited comment on HBASE-5615 at 4/5/12 4:09 PM: --- @Xufeng bq.@Rama bq .I will check the TRUNK and 0.92 version. Did u check for the trunk and 0.92? was (Author: ram_krish): @Xufeng bq.@Rama I will check the TRUNK and 0.92 version. Did u check for the trunk and 0.92? the master never does balance because of balancing the parent region Key: HBASE-5615 URL: https://issues.apache.org/jira/browse/HBASE-5615 Project: HBase Issue Type: Bug Affects Versions: 0.90.7 Reporter: xufeng Assignee: xufeng Priority: Critical Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: 5615-trunk.txt, HBASE-5615-90.patch, HBASE-5615.patch, NoPatched-surefire-report-5615-90.html, Patched_surefire-report-5615-90.html the master never do balance becauseof when master do rebuildUserRegions()，it will add the parent region into AssignmentManager#servers, if balancer let the parent region to move,the parent will in RIT forever.thus balance will never be executed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5708) [89-fb] Make MiniMapRedCluster directory a subdirectory of target/test
[ https://issues.apache.org/jira/browse/HBASE-5708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247328#comment-13247328 ] Phabricator commented on HBASE-5708: mbautin has commented on the revision [jira] [HBASE-5708] [89-fb] Improving robustness of map-reduce-related and other unit tests. Here are some more results after running every test 10 times: - Before the patch: 76 test errors, 62 test failures, 17 tests without results - After the patch: 20 test errors, 22 test failures, 24 tests without results (2.34x or 57% reduction in total failures). The remaining tests will probably take some more thinking to fix. REVISION DETAIL https://reviews.facebook.net/D2601 [89-fb] Make MiniMapRedCluster directory a subdirectory of target/test -- Key: HBASE-5708 URL: https://issues.apache.org/jira/browse/HBASE-5708 Project: HBase Issue Type: Bug Reporter: Mikhail Bautin Priority: Minor Attachments: D2601.1.patch, D2601.2.patch, D2601.3.patch Some map-reduce-based tests are failing when executed concurrently in 89-fb because mini-map-reduce cluster uses /tmp/hadoop-username for temporary data. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5727) secure hbase build broke because of 'HBASE-5451 Switch RPC call envelope/headers to PBs'
[ https://issues.apache.org/jira/browse/HBASE-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247329#comment-13247329 ] Devaraj Das commented on HBASE-5727: I will get to it today. Sorry that I broke the build. secure hbase build broke because of 'HBASE-5451 Switch RPC call envelope/headers to PBs' Key: HBASE-5727 URL: https://issues.apache.org/jira/browse/HBASE-5727 Project: HBase Issue Type: Bug Reporter: stack Assignee: Devaraj Das Priority: Blocker If you build with the security profile -- i.e. add '-P security' on the command line -- you'll see that the secure build is broke since we messed in rpc. Assigning Deveraj to take a look. If you can't work on this now DD, just give it back to me and I'll have a go at it. Thanks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5721) Update bundled hadoop to be 1.0.2 (it was just released)
[ https://issues.apache.org/jira/browse/HBASE-5721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247330#comment-13247330 ] Lars Hofhansl commented on HBASE-5721: -- HADOOP-7206 was integrated into 1.0.2. That might change how snappy would be setup for HBase. I will try that out today. Update bundled hadoop to be 1.0.2 (it was just released) Key: HBASE-5721 URL: https://issues.apache.org/jira/browse/HBASE-5721 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Fix For: 0.94.0 Attachments: 1.0.2.txt, 5721.txt, 5721.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5724) Row cache of KeyValue should be cleared in readFields().
[ https://issues.apache.org/jira/browse/HBASE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247338#comment-13247338 ] Lars Hofhansl commented on HBASE-5724: -- Good find +1 Row cache of KeyValue should be cleared in readFields(). Key: HBASE-5724 URL: https://issues.apache.org/jira/browse/HBASE-5724 Project: HBase Issue Type: Bug Affects Versions: 0.92.1 Reporter: Teruyoshi Zenmyo Assignee: Teruyoshi Zenmyo Fix For: 0.94.0, 0.96.0 Attachments: HBASE-5724.txt KeyValue does not clear its row cache in reading new values (readFields()). Therefore, If a KeyValue (kv) which caches its row bytes reads another KeyValue instance, kv.getRow() returns a wrong value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5724) Row cache of KeyValue should be cleared in readFields().
[ https://issues.apache.org/jira/browse/HBASE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247341#comment-13247341 ] stack commented on HBASE-5724: -- I can fix the comment on commit... Row cache of KeyValue should be cleared in readFields(). Key: HBASE-5724 URL: https://issues.apache.org/jira/browse/HBASE-5724 Project: HBase Issue Type: Bug Affects Versions: 0.92.1 Reporter: Teruyoshi Zenmyo Assignee: Teruyoshi Zenmyo Fix For: 0.94.0, 0.96.0 Attachments: HBASE-5724.txt KeyValue does not clear its row cache in reading new values (readFields()). Therefore, If a KeyValue (kv) which caches its row bytes reads another KeyValue instance, kv.getRow() returns a wrong value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5715) Revert 'Instant schema alter' for now, HBASE-4213
[ https://issues.apache.org/jira/browse/HBASE-5715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247347#comment-13247347 ] Subbu M Iyer commented on HBASE-5715: - Lars: I have two issues to address (Please add more if I missed any) 1. MonitoredTask usage is incorrect and needs a revisit. 2. Introduce throttling during the Region open/close requests. Given my current schedules i would imagine that this will be couple of days of effort. I would like to understand how are we going to address like track it as part of original Jira or create new one? Patches will be based on trunk or branch et al. Revert 'Instant schema alter' for now, HBASE-4213 - Key: HBASE-5715 URL: https://issues.apache.org/jira/browse/HBASE-5715 Project: HBase Issue Type: Task Reporter: stack Attachments: revert.txt, revert.v2.txt, revert.v3.txt, revert.v4.txt See this discussion: http://search-hadoop.com/m/NxCQh1KlSxR1/Pull+instant+schema+updating+out%253Fsubj=Pull+instant+schema+updating+out+ Pull out hbase-4213 for now. Can add it back later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5715) Revert 'Instant schema alter' for now, HBASE-4213
[ https://issues.apache.org/jira/browse/HBASE-5715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247349#comment-13247349 ] Subbu M Iyer commented on HBASE-5715: - Stack: Could you please help me with a branch for this as this definitely needs some baking time before we feel comfortable on a suitable release branch ? Revert 'Instant schema alter' for now, HBASE-4213 - Key: HBASE-5715 URL: https://issues.apache.org/jira/browse/HBASE-5715 Project: HBase Issue Type: Task Reporter: stack Attachments: revert.txt, revert.v2.txt, revert.v3.txt, revert.v4.txt See this discussion: http://search-hadoop.com/m/NxCQh1KlSxR1/Pull+instant+schema+updating+out%253Fsubj=Pull+instant+schema+updating+out+ Pull out hbase-4213 for now. Can add it back later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5715) Revert 'Instant schema alter' for now, HBASE-4213
[ https://issues.apache.org/jira/browse/HBASE-5715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247354#comment-13247354 ] stack commented on HBASE-5715: -- @Subbu Add testing and doc. to the list. I made you a branch for playing on here: https://svn.apache.org/repos/asf/hbase/branches/instant_schema_alter One of us committers will need to do the commits for you but thats no problem. Thanks boss. Revert 'Instant schema alter' for now, HBASE-4213 - Key: HBASE-5715 URL: https://issues.apache.org/jira/browse/HBASE-5715 Project: HBase Issue Type: Task Reporter: stack Attachments: revert.txt, revert.v2.txt, revert.v3.txt, revert.v4.txt See this discussion: http://search-hadoop.com/m/NxCQh1KlSxR1/Pull+instant+schema+updating+out%253Fsubj=Pull+instant+schema+updating+out+ Pull out hbase-4213 for now. Can add it back later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5721) Update bundled hadoop to be 1.0.2 (it was just released)
[ https://issues.apache.org/jira/browse/HBASE-5721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247357#comment-13247357 ] Lars Hofhansl commented on HBASE-5721: -- Still works fine, now only the native snappy libraries need to be available somewhere. So +1 Update bundled hadoop to be 1.0.2 (it was just released) Key: HBASE-5721 URL: https://issues.apache.org/jira/browse/HBASE-5721 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Fix For: 0.94.0 Attachments: 1.0.2.txt, 5721.txt, 5721.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5715) Revert 'Instant schema alter' for now, HBASE-4213
[ https://issues.apache.org/jira/browse/HBASE-5715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247361#comment-13247361 ] Lars Hofhansl commented on HBASE-5715: -- @Subbu: I might have some spare cycles to help with this. Revert 'Instant schema alter' for now, HBASE-4213 - Key: HBASE-5715 URL: https://issues.apache.org/jira/browse/HBASE-5715 Project: HBase Issue Type: Task Reporter: stack Attachments: revert.txt, revert.v2.txt, revert.v3.txt, revert.v4.txt See this discussion: http://search-hadoop.com/m/NxCQh1KlSxR1/Pull+instant+schema+updating+out%253Fsubj=Pull+instant+schema+updating+out+ Pull out hbase-4213 for now. Can add it back later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5715) Revert 'Instant schema alter' for now, HBASE-4213
[ https://issues.apache.org/jira/browse/HBASE-5715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247369#comment-13247369 ] stack commented on HBASE-5715: -- I'm going to commit this in next few hours unless objection, to trunk and 0.94. Revert 'Instant schema alter' for now, HBASE-4213 - Key: HBASE-5715 URL: https://issues.apache.org/jira/browse/HBASE-5715 Project: HBase Issue Type: Task Reporter: stack Attachments: revert.txt, revert.v2.txt, revert.v3.txt, revert.v4.txt See this discussion: http://search-hadoop.com/m/NxCQh1KlSxR1/Pull+instant+schema+updating+out%253Fsubj=Pull+instant+schema+updating+out+ Pull out hbase-4213 for now. Can add it back later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5715) Revert 'Instant schema alter' for now, HBASE-4213
[ https://issues.apache.org/jira/browse/HBASE-5715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247371#comment-13247371 ] Zhihong Yu commented on HBASE-5715: --- @Subbu: I will review your patches and run through test cases. Revert 'Instant schema alter' for now, HBASE-4213 - Key: HBASE-5715 URL: https://issues.apache.org/jira/browse/HBASE-5715 Project: HBase Issue Type: Task Reporter: stack Attachments: revert.txt, revert.v2.txt, revert.v3.txt, revert.v4.txt See this discussion: http://search-hadoop.com/m/NxCQh1KlSxR1/Pull+instant+schema+updating+out%253Fsubj=Pull+instant+schema+updating+out+ Pull out hbase-4213 for now. Can add it back later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5720) HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums
[ https://issues.apache.org/jira/browse/HBASE-5720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247373#comment-13247373 ] dhruba borthakur commented on HBASE-5720: - +1 to this patch. I think this bug exists in trunk too. Matt, thanks for finding it. Have you tried this patch in your setup to verify that it fixes the problem for 0.94? If so, i will submit a patch for trunk too. HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums -- Key: HBASE-5720 URL: https://issues.apache.org/jira/browse/HBASE-5720 Project: HBase Issue Type: Bug Components: io, regionserver Affects Versions: 0.94.0 Reporter: Matt Corgan Priority: Blocker Fix For: 0.94.0 Attachments: HBASE-5720-v1.patch When reading a .92 HFile without checksums, encoding it, and storing in the block cache, the HFileDataBlockEncoderImpl always allocates a dummy header appropriate for checksums even though there are none. This corrupts the byte[]. Attaching a patch that allocates a DUMMY_HEADER_NO_CHECKSUM in that case which I think is the desired behavior. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5729) Jenkins build failing; failsafe NPE'ing
Jenkins build failing; failsafe NPE'ing --- Key: HBASE-5729 URL: https://issues.apache.org/jira/browse/HBASE-5729 Project: HBase Issue Type: Bug Reporter: stack Priority: Blocker Builds up on jenkins have been failing over the last few days. Looking at it w/ nkeyway, its kinda odd. I ran exact command locally as did N and it works fine. I removed all of my repo and still works. N looked at surefire source. Its the includes that is coming back empty causing the NPE we see up on jenkins. Extra odd is that it does not seem like it a checkin of ours that brought this on. See here where its 'working' on 0.94 branch: https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.94/76/ Then a little later Ted triggers a build w/ no changes made: https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.94/77/console Its failing running the integration test phase. Let me mess around and try and get it going again. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5720) HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums
[ https://issues.apache.org/jira/browse/HBASE-5720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247378#comment-13247378 ] Matt Corgan commented on HBASE-5720: I've confirmed in some isolated test cases, and I'm running the test suite now. I think i have one more fix as well though where .92 files cannot get encoded from disk to memory because the .92 files have no encoderId. Doesn't break anything but skips the desired encoding. I'll add that fix to this patch since they're both related to .92 compatibility. HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums -- Key: HBASE-5720 URL: https://issues.apache.org/jira/browse/HBASE-5720 Project: HBase Issue Type: Bug Components: io, regionserver Affects Versions: 0.94.0 Reporter: Matt Corgan Priority: Blocker Fix For: 0.94.0 Attachments: HBASE-5720-v1.patch When reading a .92 HFile without checksums, encoding it, and storing in the block cache, the HFileDataBlockEncoderImpl always allocates a dummy header appropriate for checksums even though there are none. This corrupts the byte[]. Attaching a patch that allocates a DUMMY_HEADER_NO_CHECKSUM in that case which I think is the desired behavior. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5721) Update bundled hadoop to be 1.0.2 (it was just released)
[ https://issues.apache.org/jira/browse/HBASE-5721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247386#comment-13247386 ] Hadoop QA commented on HBASE-5721: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12521526/5721.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster org.apache.hadoop.hbase.io.hfile.TestLruBlockCache Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1405//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1405//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1405//console This message is automatically generated. Update bundled hadoop to be 1.0.2 (it was just released) Key: HBASE-5721 URL: https://issues.apache.org/jira/browse/HBASE-5721 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Fix For: 0.94.0 Attachments: 1.0.2.txt, 5721.txt, 5721.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5729) Jenkins build failing; failsafe NPE'ing
[ https://issues.apache.org/jira/browse/HBASE-5729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247387#comment-13247387 ] stack commented on HBASE-5729: -- Odd is that hadoopqa doesn't have this problem. Jenkins build failing; failsafe NPE'ing --- Key: HBASE-5729 URL: https://issues.apache.org/jira/browse/HBASE-5729 Project: HBase Issue Type: Bug Reporter: stack Priority: Blocker Builds up on jenkins have been failing over the last few days. Looking at it w/ nkeyway, its kinda odd. I ran exact command locally as did N and it works fine. I removed all of my repo and still works. N looked at surefire source. Its the includes that is coming back empty causing the NPE we see up on jenkins. Extra odd is that it does not seem like it a checkin of ours that brought this on. See here where its 'working' on 0.94 branch: https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.94/76/ Then a little later Ted triggers a build w/ no changes made: https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.94/77/console Its failing running the integration test phase. Let me mess around and try and get it going again. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5726) TestSplitTransactionOnCluster occasionally failing
[
https://issues.apache.org/jira/browse/HBASE-5726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247388#comment-13247388
]
Zhihong Yu commented on HBASE-5726:
---
@Uma:
Can you attach test output to this issue ?
Was this seen on trunk build ?
TestSplitTransactionOnCluster occasionally failing
--
Key: HBASE-5726
URL: https://issues.apache.org/jira/browse/HBASE-5726
Project: HBase
Issue Type: Bug
Reporter: Uma Maheswara Rao G
When I ran TestSplitTransactionOnCluster, some times tests are failing.
{quote}
java.lang.AssertionError: expected:1 but was:0
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.getAndCheckSingleTableRegion(TestSplitTransactionOnCluster.java:89)
at
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testShutdownFixupWhenDaughterHasSplit(TestSplitTransactionOnCluster.java:298)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
{quote}
Seems like test is flaky, random other cases also fails.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5451) Switch RPC call envelope/headers to PBs
[ https://issues.apache.org/jira/browse/HBASE-5451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247399#comment-13247399 ] Gary Helmling commented on HBASE-5451: -- This change completely breaks SecureRpcEngine, which extended HBaseClient, HBaseServer, ConnectionHeader to share common functionality. I think any changes to the RPC layer should be tested with -P security as well as without. The org.apache.hadoop.hbase.security.User.createUser() addition also leaks out the previous User encapsulation of secure vs. non-secure UGI usage. It seemed the majority was in favor of requiring Hadoop 1.0+ for HBase 0.96 in the dev list discussion, so I'm not sure this is a major issue. But seems like it would be better for consistency to push the UGI calls down in to User.SecureHadoopUser. Or we could discuss removing User.HadoopUser as cleanup in 0.96 if it's no longer supported. Is there a plan with how the move the PB's integrates with security? Switch RPC call envelope/headers to PBs --- Key: HBASE-5451 URL: https://issues.apache.org/jira/browse/HBASE-5451 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Affects Versions: 0.94.0 Reporter: Todd Lipcon Assignee: Devaraj Das Fix For: 0.96.0 Attachments: 5305v7.txt, 5305v7.txt, rpc-proto.2.txt, rpc-proto.3.txt, rpc-proto.patch.1_2, rpc-proto.r5.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5721) Update bundled hadoop to be 1.0.2 (it was just released)
[ https://issues.apache.org/jira/browse/HBASE-5721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5721: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I ran the two failed tests locally and they passed. Applied to 0.94 and to trunk. Update bundled hadoop to be 1.0.2 (it was just released) Key: HBASE-5721 URL: https://issues.apache.org/jira/browse/HBASE-5721 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Fix For: 0.94.0 Attachments: 1.0.2.txt, 5721.txt, 5721.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5729) Jenkins build failing; failsafe NPE'ing
[
https://issues.apache.org/jira/browse/HBASE-5729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247405#comment-13247405
]
stack commented on HBASE-5729:
--
So, I removed the '-P release' flag from 0.94 and it started building again.
0.92 has the flag and it does not seem to be failing.
{code}
-e -X clean -Dmaven.test.redirectTestOutputToFile=true site install
assembly:single -Prelease
{code}
Its where I copied the flag from.
Maybe needs to be '-Prelease' instead of '-P release' as I had.
Let me try on trunk... I've not changed that yet.
Jenkins build failing; failsafe NPE'ing
---
Key: HBASE-5729
URL: https://issues.apache.org/jira/browse/HBASE-5729
Project: HBase
Issue Type: Bug
Reporter: stack
Priority: Blocker
Builds up on jenkins have been failing over the last few days. Looking at it
w/ nkeyway, its kinda odd. I ran exact command locally as did N and it works
fine. I removed all of my repo and still works. N looked at surefire
source. Its the includes that is coming back empty causing the NPE we see up
on jenkins. Extra odd is that it does not seem like it a checkin of ours
that brought this on. See here where its 'working' on 0.94 branch:
https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.94/76/ Then a
little later Ted triggers a build w/ no changes made:
https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.94/77/console
Its failing running the integration test phase. Let me mess around and try
and get it going again.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5729) Jenkins build failing; failsafe NPE'ing
[ https://issues.apache.org/jira/browse/HBASE-5729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247412#comment-13247412 ] stack commented on HBASE-5729: -- hmmm... no I had '-Prelease'. Will try same version of surefire the integration tests use in 0.92 in 0.94 next Jenkins build failing; failsafe NPE'ing --- Key: HBASE-5729 URL: https://issues.apache.org/jira/browse/HBASE-5729 Project: HBase Issue Type: Bug Reporter: stack Priority: Blocker Builds up on jenkins have been failing over the last few days. Looking at it w/ nkeyway, its kinda odd. I ran exact command locally as did N and it works fine. I removed all of my repo and still works. N looked at surefire source. Its the includes that is coming back empty causing the NPE we see up on jenkins. Extra odd is that it does not seem like it a checkin of ours that brought this on. See here where its 'working' on 0.94 branch: https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.94/76/ Then a little later Ted triggers a build w/ no changes made: https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.94/77/console Its failing running the integration test phase. Let me mess around and try and get it going again. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5217) Reenable the thrift tests, and add a new one for getRegionInfo
[ https://issues.apache.org/jira/browse/HBASE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Newman updated HBASE-5217: --- Status: Patch Available (was: In Progress) Reenable the thrift tests, and add a new one for getRegionInfo -- Key: HBASE-5217 URL: https://issues.apache.org/jira/browse/HBASE-5217 Project: HBase Issue Type: Improvement Reporter: Alex Newman Assignee: Alex Newman Priority: Minor Attachments: 0001-Fixing-thrift-tests-v2.patch, 0001-Fixing-thrift-tests.patch, 0002-HBASE-5217.-Reenable-the-thrift-tests-and-add-a-new-.patch, -hbase-posix4e #92 Console [Jenkins].pdf At some point we disabled tests for the thrift server. In addition, it looks like the getRegionInfo no longer functions. I'd like to reenable the tests and add one for getRegionInfo. I had to write this to test my changes in HBASE-2600 anyway. I figured I would break it out. We shouldn't commit it until we have fixed getting the regioninfo from the thriftserver. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5729) Jenkins build failing; failsafe NPE'ing
[
https://issues.apache.org/jira/browse/HBASE-5729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247414#comment-13247414
]
nkeywal commented on HBASE-5729:
locally, I reproduce the issue with
{noformat}
mvn -e -X site install assembly:single
{noformat}
Jenkins build failing; failsafe NPE'ing
---
Key: HBASE-5729
URL: https://issues.apache.org/jira/browse/HBASE-5729
Project: HBase
Issue Type: Bug
Reporter: stack
Priority: Blocker
Builds up on jenkins have been failing over the last few days. Looking at it
w/ nkeyway, its kinda odd. I ran exact command locally as did N and it works
fine. I removed all of my repo and still works. N looked at surefire
source. Its the includes that is coming back empty causing the NPE we see up
on jenkins. Extra odd is that it does not seem like it a checkin of ours
that brought this on. See here where its 'working' on 0.94 branch:
https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.94/76/ Then a
little later Ted triggers a build w/ no changes made:
https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.94/77/console
Its failing running the integration test phase. Let me mess around and try
and get it going again.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2600) Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid
[ https://issues.apache.org/jira/browse/HBASE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247416#comment-13247416 ] Alex Newman commented on HBASE-2600: What do I need to do to move this forward? Change how we do meta tables; from tablename+STARTROW+randomid to instead, tablename+ENDROW+randomid Key: HBASE-2600 URL: https://issues.apache.org/jira/browse/HBASE-2600 Project: HBase Issue Type: Bug Reporter: stack Assignee: Alex Newman Attachments: 0001-Changed-regioninfo-format-to-use-endKey-instead-of-s.patch, 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v2.patch, 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v4.patch, 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v6.patch, 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v7.2.patch, 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8, 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v8.1, 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen-v9.patch, 0001-HBASE-2600.-Change-how-we-do-meta-tables-from-tablen.patch, 2600-trunk-01-17.txt, HBASE-2600+5217-Sun-Mar-25-2012-v3.patch, HBASE-2600+5217-Sun-Mar-25-2012-v4.patch, hbase-2600-root.dir.tgz, jenkins.pdf This is an idea that Ryan and I have been kicking around on and off for a while now. If regionnames were made of tablename+endrow instead of tablename+startrow, then in the metatables, doing a search for the region that contains the wanted row, we'd just have to open a scanner using passed row and the first row found by the scan would be that of the region we need (If offlined parent, we'd have to scan to the next row). If we redid the meta tables in this format, we'd be using an access that is natural to hbase, a scan as opposed to the perverse, expensive getClosestRowBefore we currently have that has to walk backward in meta finding a containing region. This issue is about changing the way we name regions. If we were using scans, prewarming client cache would be near costless (as opposed to what we'll currently have to do which is first a getClosestRowBefore and then a scan from the closestrowbefore forward). Converting to the new method, we'd have to run a migration on startup changing the content in meta. Up to this, the randomid component of a region name has been the timestamp of region creation. HBASE-2531 32-bit encoding of regionnames waaay too susceptible to hash clashes proposes changing the randomid so that it contains actual name of the directory in the filesystem that hosts the region. If we had this in place, I think it would help with the migration to this new way of doing the meta because as is, the region name in fs is a hash of regionname... changing the format of the regionname would mean we generate a different hash... so we'd need hbase-2531 to be in place before we could do this change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5451) Switch RPC call envelope/headers to PBs
[ https://issues.apache.org/jira/browse/HBASE-5451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247421#comment-13247421 ] stack commented on HBASE-5451: -- bq. Is there a plan with how the move the PB's integrates with security? No other than we don't want to break it (we should have been more proactive around security changes G). Switch RPC call envelope/headers to PBs --- Key: HBASE-5451 URL: https://issues.apache.org/jira/browse/HBASE-5451 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Affects Versions: 0.94.0 Reporter: Todd Lipcon Assignee: Devaraj Das Fix For: 0.96.0 Attachments: 5305v7.txt, 5305v7.txt, rpc-proto.2.txt, rpc-proto.3.txt, rpc-proto.patch.1_2, rpc-proto.r5.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5724) Row cache of KeyValue should be cleared in readFields().
[ https://issues.apache.org/jira/browse/HBASE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247422#comment-13247422 ] Todd Lipcon commented on HBASE-5724: Does this need to go back to 0.90.x as well? Row cache of KeyValue should be cleared in readFields(). Key: HBASE-5724 URL: https://issues.apache.org/jira/browse/HBASE-5724 Project: HBase Issue Type: Bug Affects Versions: 0.92.1 Reporter: Teruyoshi Zenmyo Assignee: Teruyoshi Zenmyo Fix For: 0.94.0, 0.96.0 Attachments: HBASE-5724.txt KeyValue does not clear its row cache in reading new values (readFields()). Therefore, If a KeyValue (kv) which caches its row bytes reads another KeyValue instance, kv.getRow() returns a wrong value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5729) Jenkins build failing; failsafe NPE'ing
[ https://issues.apache.org/jira/browse/HBASE-5729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247425#comment-13247425 ] stack commented on HBASE-5729: -- I added back ' -DskipITs -Prelease' at N's suggestion (seems to fix his local breakage). Jenkins seems totally hosed at mo so letting this be a w hile. Jenkins build failing; failsafe NPE'ing --- Key: HBASE-5729 URL: https://issues.apache.org/jira/browse/HBASE-5729 Project: HBase Issue Type: Bug Reporter: stack Priority: Blocker Builds up on jenkins have been failing over the last few days. Looking at it w/ nkeyway, its kinda odd. I ran exact command locally as did N and it works fine. I removed all of my repo and still works. N looked at surefire source. Its the includes that is coming back empty causing the NPE we see up on jenkins. Extra odd is that it does not seem like it a checkin of ours that brought this on. See here where its 'working' on 0.94 branch: https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.94/76/ Then a little later Ted triggers a build w/ no changes made: https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.94/77/console Its failing running the integration test phase. Let me mess around and try and get it going again. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5730) [89-fb] Make HRegionThriftServer's thread pool bounded
[ https://issues.apache.org/jira/browse/HBASE-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HBASE-5730: -- Description: This JIRA is for a quick fix in 89-fb to reuse TBoundedThreadPoolServer in HRegionThriftServer. We will address whatever problems HRegionThriftServer still has in trunk in HBASE-5703. [89-fb] Make HRegionThriftServer's thread pool bounded -- Key: HBASE-5730 URL: https://issues.apache.org/jira/browse/HBASE-5730 Project: HBase Issue Type: Bug Reporter: Mikhail Bautin Assignee: Mikhail Bautin This JIRA is for a quick fix in 89-fb to reuse TBoundedThreadPoolServer in HRegionThriftServer. We will address whatever problems HRegionThriftServer still has in trunk in HBASE-5703. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5729) Jenkins build failing; failsafe NPE'ing
[ https://issues.apache.org/jira/browse/HBASE-5729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247434#comment-13247434 ] stack commented on HBASE-5729: -- The last addition seems to work for trunk. Let me see about 0.94. Jenkins build failing; failsafe NPE'ing --- Key: HBASE-5729 URL: https://issues.apache.org/jira/browse/HBASE-5729 Project: HBase Issue Type: Bug Reporter: stack Priority: Blocker Builds up on jenkins have been failing over the last few days. Looking at it w/ nkeyway, its kinda odd. I ran exact command locally as did N and it works fine. I removed all of my repo and still works. N looked at surefire source. Its the includes that is coming back empty causing the NPE we see up on jenkins. Extra odd is that it does not seem like it a checkin of ours that brought this on. See here where its 'working' on 0.94 branch: https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.94/76/ Then a little later Ted triggers a build w/ no changes made: https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.94/77/console Its failing running the integration test phase. Let me mess around and try and get it going again. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5217) Reenable the thrift tests, and add a new one for getRegionInfo
[ https://issues.apache.org/jira/browse/HBASE-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247432#comment-13247432 ] Hadoop QA commented on HBASE-5217: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12521535/0002-HBASE-5217-v3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1406//console This message is automatically generated. Reenable the thrift tests, and add a new one for getRegionInfo -- Key: HBASE-5217 URL: https://issues.apache.org/jira/browse/HBASE-5217 Project: HBase Issue Type: Improvement Reporter: Alex Newman Assignee: Alex Newman Priority: Minor Attachments: 0001-Fixing-thrift-tests-v2.patch, 0001-Fixing-thrift-tests.patch, 0002-HBASE-5217-v3.patch, 0002-HBASE-5217.-Reenable-the-thrift-tests-and-add-a-new-.patch, -hbase-posix4e #92 Console [Jenkins].pdf At some point we disabled tests for the thrift server. In addition, it looks like the getRegionInfo no longer functions. I'd like to reenable the tests and add one for getRegionInfo. I had to write this to test my changes in HBASE-2600 anyway. I figured I would break it out. We shouldn't commit it until we have fixed getting the regioninfo from the thriftserver. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5723) Simple Design of Secondary Index
[ https://issues.apache.org/jira/browse/HBASE-5723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247438#comment-13247438 ] Todd Lipcon commented on HBASE-5723: The design doc doesn't make any reference to consistency. How do you ensure consistency of the index? To do proper indexes you really need cross-table transactions, or at least a mechanism for eventual consistency if that's all you plan on providing. Simple Design of Secondary Index Key: HBASE-5723 URL: https://issues.apache.org/jira/browse/HBASE-5723 Project: HBase Issue Type: New Feature Components: coprocessors Reporter: ShiXing Priority: Minor Attachments: Simple Design of HBase SecondaryIndex.pdf Use coprocessor to create index. And primary tables' compaction to purge the stale data. Attach file is the Design of the Seconday Index. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs
[ https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247444#comment-13247444 ] Eran Kutner commented on HBASE-3996: @stack: I believe the only open issue in the review board is your suggestion to replace my MultiTableInputCollection with a ListScan. Although I agree it would make the patch simpler and allow it to have one less class, I think it will make using it less natural. Developers will have to create a Scan which is a common object and then set a table attribute. This feels less natural to me than setting the table by adding to a collection the way I've done it, but I guess it's a matter of perspective. Support multiple tables and scanners as input to the mapper in map/reduce jobs -- Key: HBASE-3996 URL: https://issues.apache.org/jira/browse/HBASE-3996 Project: HBase Issue Type: Improvement Components: mapreduce Reporter: Eran Kutner Assignee: Eran Kutner Fix For: 0.96.0 Attachments: 3996-v2.txt, 3996-v3.txt, 3996-v4.txt, 3996-v5.txt, 3996-v6.txt, 3996-v7.txt, HBase-3996.patch It seems that in many cases feeding data from multiple tables or multiple scanners on a single table can save a lot of time when running map/reduce jobs. I propose a new MultiTableInputFormat class that would allow doing this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5716) Make HBASE-4608 easier to use
[
https://issues.apache.org/jira/browse/HBASE-5716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247445#comment-13247445
]
Jean-Daniel Cryans commented on HBASE-5716:
---
bq. Was there a great correlation between size on disk and size of blog?
I'm not sure I understand the question.
Make HBASE-4608 easier to use
-
Key: HBASE-5716
URL: https://issues.apache.org/jira/browse/HBASE-5716
Project: HBase
Issue Type: Improvement
Reporter: Jean-Daniel Cryans
Assignee: Li Pi
Fix For: 0.96.0, 0.94.1
HBASE-4608 is a nice feature but after playing with it for a while I think
the following should be fixed to make it easier to use by someone who's not a
dev:
- Add some signal that says that the feature is turned on. Right now you can
{{jstack | grep KeyValueCompression}} a couple of times and if you get a hit
you definitely know it's on, but otherwise the random user wouldn't know
without going through the jira.
- Add documentation in the reference guide. At the minimum add
{{hbase.regionserver.wal.enablecompression}} in there with a small
description. Better would be to add a section in {{Appendix B}} or something
like that and describe the functionality a bit and who it's useful for. For
example, flush from your brain the knowledge of the patch and read the name
of the configuration... now let's say you have a use case that involves
writing easily compressible values. Any normal user would believe that this
is a good tuning parameter for them, but it's just going to waste CPU cycles.
- Add some metrics like we have for HFiles where you get a clue about the
compression ratio.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs
[ https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247448#comment-13247448 ] Eran Kutner commented on HBASE-3996: Just to give better reasoning why I feel it is unnatural. With my method someone using this functionality for the first time would be able to figure it out just by looking at the class names and interface definitions (using IDE auto completion for example), while the only way to know it is required to set that attribute is to dig in the documentation. Support multiple tables and scanners as input to the mapper in map/reduce jobs -- Key: HBASE-3996 URL: https://issues.apache.org/jira/browse/HBASE-3996 Project: HBase Issue Type: Improvement Components: mapreduce Reporter: Eran Kutner Assignee: Eran Kutner Fix For: 0.96.0 Attachments: 3996-v2.txt, 3996-v3.txt, 3996-v4.txt, 3996-v5.txt, 3996-v6.txt, 3996-v7.txt, HBase-3996.patch It seems that in many cases feeding data from multiple tables or multiple scanners on a single table can save a lot of time when running map/reduce jobs. I propose a new MultiTableInputFormat class that would allow doing this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5717) Scanner metrics are only reported if you get to the end of a scanner
[ https://issues.apache.org/jira/browse/HBASE-5717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247456#comment-13247456 ] Zhihong Yu commented on HBASE-5717: --- Patch v2 looks good to me. Scanner metrics are only reported if you get to the end of a scanner Key: HBASE-5717 URL: https://issues.apache.org/jira/browse/HBASE-5717 Project: HBase Issue Type: Bug Components: client, metrics Reporter: Ian Varley Priority: Minor Attachments: ClientScanner_HBASE_5717-v2.patch, ClientScanner_HBASE_5717.patch Original Estimate: 4h Remaining Estimate: 4h When you turn on Scanner Metrics, the metrics are currently only made available if you run over all records available in the scanner. If you stop iterating before the end, the values are never flushed into the metrics object (in the Scan attribute). Will supply a patch with fix and test. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5717) Scanner metrics are only reported if you get to the end of a scanner
[ https://issues.apache.org/jira/browse/HBASE-5717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247457#comment-13247457 ] jirapos...@reviews.apache.org commented on HBASE-5717: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4640/ --- (Updated 2012-04-05 18:15:41.419224) Review request for hbase. Changes --- Added finally block (with try/catch inside). I'd prefer to also add logging rather than swallowing the exceptions, but that seems like it should be a different Jira (and maybe should cover all cases that swallow exceptions). Also: sorry for a) sending a review on the board for something that's probably too small, and b) writing comments on my own review (intended to annotate, didn't realize it would appear as if I were a separate reviewer). Summary --- Fix for persistence of scan metrics when the scanner doesn't run to exhaustion. This addresses bug HBASE-5717. https://issues.apache.org/jira/browse/HBASE-5717 Diffs (updated) - /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1309585 /src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java 1309585 Diff: https://reviews.apache.org/r/4640/diff Testing --- Altered the scan metrics unit test to show this problem (now fails without changes to ClientScanner.java). Thanks, Ian Scanner metrics are only reported if you get to the end of a scanner Key: HBASE-5717 URL: https://issues.apache.org/jira/browse/HBASE-5717 Project: HBase Issue Type: Bug Components: client, metrics Reporter: Ian Varley Priority: Minor Attachments: ClientScanner_HBASE_5717-v2.patch, ClientScanner_HBASE_5717.patch Original Estimate: 4h Remaining Estimate: 4h When you turn on Scanner Metrics, the metrics are currently only made available if you run over all records available in the scanner. If you stop iterating before the end, the values are never flushed into the metrics object (in the Scan attribute). Will supply a patch with fix and test. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-5451) Switch RPC call envelope/headers to PBs
[ https://issues.apache.org/jira/browse/HBASE-5451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247460#comment-13247460 ] Andrew Purtell edited comment on HBASE-5451 at 4/5/12 6:25 PM: --- This patch should be reverted until the changes are refactored for sharing between the RPC engines. Edit: Subclassing of RPC engines was introduced to deal with incompatibilities in security related classes between Hadoop versions. If we are specifying Hadoop 1.0 (or equivalent shims) as a build requirement, then we could instead use a single RPC envelope and the User abstraction to paper over the remaining differences. was (Author: apurtell): This patch should be reverted until the changes are refactored for sharing between the RPC engines. Switch RPC call envelope/headers to PBs --- Key: HBASE-5451 URL: https://issues.apache.org/jira/browse/HBASE-5451 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Affects Versions: 0.94.0 Reporter: Todd Lipcon Assignee: Devaraj Das Fix For: 0.96.0 Attachments: 5305v7.txt, 5305v7.txt, rpc-proto.2.txt, rpc-proto.3.txt, rpc-proto.patch.1_2, rpc-proto.r5.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5717) Scanner metrics are only reported if you get to the end of a scanner
[ https://issues.apache.org/jira/browse/HBASE-5717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247466#comment-13247466 ] jirapos...@reviews.apache.org commented on HBASE-5717: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4640/#review6715 --- Ship it! The only comment I have is that there is now no test that verifies that metrics are collected when we scanned all the way to the end, but did not close the scanner. - Lars On 2012-04-05 18:15:41, Ian Varley wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/4640/ bq. --- bq. bq. (Updated 2012-04-05 18:15:41) bq. bq. bq. Review request for hbase. bq. bq. bq. Summary bq. --- bq. bq. Fix for persistence of scan metrics when the scanner doesn't run to exhaustion. bq. bq. bq. This addresses bug HBASE-5717. bq. https://issues.apache.org/jira/browse/HBASE-5717 bq. bq. bq. Diffs bq. - bq. bq./src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1309585 bq./src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java 1309585 bq. bq. Diff: https://reviews.apache.org/r/4640/diff bq. bq. bq. Testing bq. --- bq. bq. Altered the scan metrics unit test to show this problem (now fails without changes to ClientScanner.java). bq. bq. bq. Thanks, bq. bq. Ian bq. bq. Scanner metrics are only reported if you get to the end of a scanner Key: HBASE-5717 URL: https://issues.apache.org/jira/browse/HBASE-5717 Project: HBase Issue Type: Bug Components: client, metrics Reporter: Ian Varley Priority: Minor Attachments: ClientScanner_HBASE_5717-v2.patch, ClientScanner_HBASE_5717.patch Original Estimate: 4h Remaining Estimate: 4h When you turn on Scanner Metrics, the metrics are currently only made available if you run over all records available in the scanner. If you stop iterating before the end, the values are never flushed into the metrics object (in the Scan attribute). Will supply a patch with fix and test. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5727) secure hbase build broke because of 'HBASE-5451 Switch RPC call envelope/headers to PBs'
[ https://issues.apache.org/jira/browse/HBASE-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247469#comment-13247469 ] Andrew Purtell commented on HBASE-5727: --- I commented over on HBASE-5451: Subclassing of RPC engines was introduced to deal with incompatibilities in security related classes between Hadoop versions. If we are specifying Hadoop 1.0 (or equivalent shims) as a build requirement, then we could instead use a single RPC envelope and the User abstraction to paper over the remaining differences. secure hbase build broke because of 'HBASE-5451 Switch RPC call envelope/headers to PBs' Key: HBASE-5727 URL: https://issues.apache.org/jira/browse/HBASE-5727 Project: HBase Issue Type: Bug Reporter: stack Assignee: Devaraj Das Priority: Blocker If you build with the security profile -- i.e. add '-P security' on the command line -- you'll see that the secure build is broke since we messed in rpc. Assigning Deveraj to take a look. If you can't work on this now DD, just give it back to me and I'll have a go at it. Thanks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5731) Make max line length 100 in linter
Make max line length 100 in linter -- Key: HBASE-5731 URL: https://issues.apache.org/jira/browse/HBASE-5731 Project: HBase Issue Type: New Feature Reporter: Mikhail Bautin Assignee: Mikhail Bautin Priority: Minor We have switched to 100 characters per line in our Java files. Making the change in the linter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5727) secure hbase build broke because of 'HBASE-5451 Switch RPC call envelope/headers to PBs'
[ https://issues.apache.org/jira/browse/HBASE-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247474#comment-13247474 ] Andrew Purtell commented on HBASE-5727: --- So, to be clear, I mean as part of this work consolidate the RPC engines assuming Hadoop 1.0 or equivalent shims. Then the -P security profile would just build coprocessors. There would be little (no?) need for a security profile at all. secure hbase build broke because of 'HBASE-5451 Switch RPC call envelope/headers to PBs' Key: HBASE-5727 URL: https://issues.apache.org/jira/browse/HBASE-5727 Project: HBase Issue Type: Bug Reporter: stack Assignee: Devaraj Das Priority: Blocker If you build with the security profile -- i.e. add '-P security' on the command line -- you'll see that the secure build is broke since we messed in rpc. Assigning Deveraj to take a look. If you can't work on this now DD, just give it back to me and I'll have a go at it. Thanks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5731) Make max line length 100 in linter
[ https://issues.apache.org/jira/browse/HBASE-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247483#comment-13247483 ] Phabricator commented on HBASE-5731: tedyu has accepted the revision [jira] [HBASE-5731] [89-fb] Make max line length 100 in linter. REVISION DETAIL https://reviews.facebook.net/D2625 BRANCH master Make max line length 100 in linter -- Key: HBASE-5731 URL: https://issues.apache.org/jira/browse/HBASE-5731 Project: HBase Issue Type: New Feature Reporter: Mikhail Bautin Assignee: Mikhail Bautin Priority: Minor Attachments: D2625.1.patch We have switched to 100 characters per line in our Java files. Making the change in the linter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5731) Make max line length 100 in linter
[ https://issues.apache.org/jira/browse/HBASE-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5731: --- Attachment: D2631.1.patch mbautin requested code review of [jira] [HBASE-5731] Make max line length 100 in linter. Reviewers: JIRA, nspiegelberg, Kannan, Liyin, tedyu, stack We have switched to 100 characters per line in our Java files. Making the change in the linter. The 89-fb version is D2625. TEST PLAN arc lint REVISION DETAIL https://reviews.facebook.net/D2631 AFFECTED FILES .arcconfig MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/6045/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. Make max line length 100 in linter -- Key: HBASE-5731 URL: https://issues.apache.org/jira/browse/HBASE-5731 Project: HBase Issue Type: New Feature Reporter: Mikhail Bautin Assignee: Mikhail Bautin Priority: Minor Attachments: D2625.1.patch, D2631.1.patch We have switched to 100 characters per line in our Java files. Making the change in the linter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5731) Make max line length 100 in linter
[ https://issues.apache.org/jira/browse/HBASE-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247493#comment-13247493 ] Phabricator commented on HBASE-5731: stack has commented on the revision [jira] [HBASE-5731] Make max line length 100 in linter. +1 REVISION DETAIL https://reviews.facebook.net/D2631 Make max line length 100 in linter -- Key: HBASE-5731 URL: https://issues.apache.org/jira/browse/HBASE-5731 Project: HBase Issue Type: New Feature Reporter: Mikhail Bautin Assignee: Mikhail Bautin Priority: Minor Attachments: D2625.1.patch, D2631.1.patch We have switched to 100 characters per line in our Java files. Making the change in the linter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5727) secure hbase build broke because of 'HBASE-5451 Switch RPC call envelope/headers to PBs'
[ https://issues.apache.org/jira/browse/HBASE-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-5727: --- Attachment: 5727.patch Attaching a patch that makes the build go through (at least compiles). I haven't run the tests with this. secure hbase build broke because of 'HBASE-5451 Switch RPC call envelope/headers to PBs' Key: HBASE-5727 URL: https://issues.apache.org/jira/browse/HBASE-5727 Project: HBase Issue Type: Bug Reporter: stack Assignee: Devaraj Das Priority: Blocker Attachments: 5727.patch If you build with the security profile -- i.e. add '-P security' on the command line -- you'll see that the secure build is broke since we messed in rpc. Assigning Deveraj to take a look. If you can't work on this now DD, just give it back to me and I'll have a go at it. Thanks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5716) Make HBASE-4608 easier to use
[
https://issues.apache.org/jira/browse/HBASE-5716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247496#comment-13247496
]
Li Pi commented on HBASE-5716:
--
Gah. Autocorrect - and me being sleepy.
What I meant is, it seems like there exist cases where log replay time and Hlog
size on disk didn't correlate well before, and there will still be some now -
has it really ever been much of an issue?
Make HBASE-4608 easier to use
-
Key: HBASE-5716
URL: https://issues.apache.org/jira/browse/HBASE-5716
Project: HBase
Issue Type: Improvement
Reporter: Jean-Daniel Cryans
Assignee: Li Pi
Fix For: 0.96.0, 0.94.1
HBASE-4608 is a nice feature but after playing with it for a while I think
the following should be fixed to make it easier to use by someone who's not a
dev:
- Add some signal that says that the feature is turned on. Right now you can
{{jstack | grep KeyValueCompression}} a couple of times and if you get a hit
you definitely know it's on, but otherwise the random user wouldn't know
without going through the jira.
- Add documentation in the reference guide. At the minimum add
{{hbase.regionserver.wal.enablecompression}} in there with a small
description. Better would be to add a section in {{Appendix B}} or something
like that and describe the functionality a bit and who it's useful for. For
example, flush from your brain the knowledge of the patch and read the name
of the configuration... now let's say you have a use case that involves
writing easily compressible values. Any normal user would believe that this
is a good tuning parameter for them, but it's just going to waste CPU cycles.
- Add some metrics like we have for HFiles where you get a clue about the
compression ratio.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5720) HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums
[ https://issues.apache.org/jira/browse/HBASE-5720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Corgan updated HBASE-5720: --- Attachment: HBASE-5720-v2.patch v2 patch also reverts the logic in HFileDataBlockEncoderImpl.createFromFileInfo to an earlier version. Version on .94 branch will not allow disk-cache encoding on a .92 because (in short) it doesn't have an encoderId. I'm pretty sure this is something we want to support, but correct me if i'm wrong. Without it you'd have to major compact everything before using encoding (or something along those lines) Test suite passing (except unrelated failure), and it works in my benchmarking setup for HBASE-4676 HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums -- Key: HBASE-5720 URL: https://issues.apache.org/jira/browse/HBASE-5720 Project: HBase Issue Type: Bug Components: io, regionserver Affects Versions: 0.94.0 Reporter: Matt Corgan Priority: Blocker Fix For: 0.94.0 Attachments: HBASE-5720-v1.patch, HBASE-5720-v2.patch When reading a .92 HFile without checksums, encoding it, and storing in the block cache, the HFileDataBlockEncoderImpl always allocates a dummy header appropriate for checksums even though there are none. This corrupts the byte[]. Attaching a patch that allocates a DUMMY_HEADER_NO_CHECKSUM in that case which I think is the desired behavior. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5313) Restructure hfiles layout for better compression
[ https://issues.apache.org/jira/browse/HBASE-5313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247497#comment-13247497 ] He Yongqiang commented on HBASE-5313: - Hi Kannan, We are still experimenting this. The initial results shows only less than one quarter off, which is kind of not big enough for us. The timestamp issue is a low hanging fruit, which can cut 8%. We will post some diff asap, once after we finalize our experiments. Restructure hfiles layout for better compression Key: HBASE-5313 URL: https://issues.apache.org/jira/browse/HBASE-5313 Project: HBase Issue Type: Improvement Components: io Reporter: dhruba borthakur Assignee: dhruba borthakur A HFile block contain a stream of key-values. Can we can organize these kvs on the disk in a better way so that we get much greater compression ratios? One option (thanks Prakash) is to store all the keys in the beginning of the block (let's call this the key-section) and then store all their corresponding values towards the end of the block. This will allow us to not-even decompress the values when we are scanning and skipping over rows in the block. Any other ideas? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5720) HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums
[ https://issues.apache.org/jira/browse/HBASE-5720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247501#comment-13247501 ] Hadoop QA commented on HBASE-5720: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12521547/HBASE-5720-v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1408//console This message is automatically generated. HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums -- Key: HBASE-5720 URL: https://issues.apache.org/jira/browse/HBASE-5720 Project: HBase Issue Type: Bug Components: io, regionserver Affects Versions: 0.94.0 Reporter: Matt Corgan Priority: Blocker Fix For: 0.94.0 Attachments: HBASE-5720-v1.patch, HBASE-5720-v2.patch When reading a .92 HFile without checksums, encoding it, and storing in the block cache, the HFileDataBlockEncoderImpl always allocates a dummy header appropriate for checksums even though there are none. This corrupts the byte[]. Attaching a patch that allocates a DUMMY_HEADER_NO_CHECKSUM in that case which I think is the desired behavior. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5731) Make max line length 100 in linter
[ https://issues.apache.org/jira/browse/HBASE-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247505#comment-13247505 ] Phabricator commented on HBASE-5731: mbautin has committed the revision [jira] [HBASE-5731] [89-fb] Make max line length 100 in linter. REVISION DETAIL https://reviews.facebook.net/D2625 COMMIT https://reviews.facebook.net/rHBASEEIGHTNINEFBBRANCH131 Make max line length 100 in linter -- Key: HBASE-5731 URL: https://issues.apache.org/jira/browse/HBASE-5731 Project: HBase Issue Type: New Feature Reporter: Mikhail Bautin Assignee: Mikhail Bautin Priority: Minor Attachments: D2625.1.patch, D2631.1.patch We have switched to 100 characters per line in our Java files. Making the change in the linter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5547) Don't delete HFiles when in backup mode
[ https://issues.apache.org/jira/browse/HBASE-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247503#comment-13247503 ] jirapos...@reviews.apache.org commented on HBASE-5547: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4633/#review6714 --- Thanks for the review Ted. Any thoughts on combining the Tracker and Manager (see inline comment below on HFileArchiveManager). src/main/java/org/apache/hadoop/hbase/HConstants.java https://reviews.apache.org/r/4633/#comment14522 fair enough. done. src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java https://reviews.apache.org/r/4633/#comment14517 Disable right now just wipes out all the tables currently in backup mode (not saving any info), but I guess it could be reasonable to want to start backing up all the tables - though in real usage you probably only want to do one table at a time to minimize impact on a cluster. I'd rather not give people that power directly, but make them jump through a couple hoops (listing out all the tables and then iteratively starting backups on those tables) due: 1) how much data that can be, and 2) in a real 'backup' context this means they are cloning all the tables (doing a full table scan on all tables) which would be highly disruptive. src/main/java/org/apache/hadoop/hbase/regionserver/HFileArchiveManager.java https://reviews.apache.org/r/4633/#comment14518 done. src/main/java/org/apache/hadoop/hbase/regionserver/HFileArchiveManager.java https://reviews.apache.org/r/4633/#comment14519 done src/main/java/org/apache/hadoop/hbase/regionserver/HFileArchiveManager.java https://reviews.apache.org/r/4633/#comment14520 I've been debating whether or not to roll this implementation into the HFileArchiveTracker and just move this into an interface. It doesn't actually do very much - essentially a prettier front to a map and could easily be moved into the tracker. Thoughts? src/main/java/org/apache/hadoop/hbase/regionserver/HFileArchiveManager.java https://reviews.apache.org/r/4633/#comment14521 done. src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java https://reviews.apache.org/r/4633/#comment14526 done. src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java https://reviews.apache.org/r/4633/#comment14523 That case should only come up in testing, so logging seems excessive. src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java https://reviews.apache.org/r/4633/#comment14524 used store because in code that's what we are really dealing with, but everyone just refers to them as hfiles. going with your way for consistency. src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java https://reviews.apache.org/r/4633/#comment14525 done. src/main/java/org/apache/hadoop/hbase/util/HFileArchiveUtil.java https://reviews.apache.org/r/4633/#comment14527 done. src/main/java/org/apache/hadoop/hbase/util/HFileArchiveUtil.java https://reviews.apache.org/r/4633/#comment14528 yup, nice catch (no pun intended). src/main/java/org/apache/hadoop/hbase/util/HFileArchiveUtil.java https://reviews.apache.org/r/4633/#comment14529 true. earlier iteration comment that got confused with manager.keepFiles src/main/java/org/apache/hadoop/hbase/util/HFileArchiveUtil.java https://reviews.apache.org/r/4633/#comment14530 how so? renames an the src path to the dest path. Seems to work in tests. src/main/java/org/apache/hadoop/hbase/util/HFileArchiveUtil.java https://reviews.apache.org/r/4633/#comment14531 then its something we don't care about - only care about the hfiles, which are stored in one directory level below the region. src/main/java/org/apache/hadoop/hbase/util/HFileArchiveUtil.java https://reviews.apache.org/r/4633/#comment14532 done. src/main/java/org/apache/hadoop/hbase/util/HFileArchiveUtil.java https://reviews.apache.org/r/4633/#comment14534 Definitely need to add another unit test covering the 'resolve' part of this method - lot of gymnastics/corner cases going on. src/main/java/org/apache/hadoop/hbase/util/HFileArchiveUtil.java https://reviews.apache.org/r/4633/#comment14536 reworked a little bit in the new version. src/main/java/org/apache/hadoop/hbase/util/HFileArchiveUtil.java https://reviews.apache.org/r/4633/#comment14533 going to bail out earlier if we can't delete the old file. src/main/java/org/apache/hadoop/hbase/util/HFileArchiveUtil.java https://reviews.apache.org/r/4633/#comment14535 done. src/main/java/org/apache/hadoop/hbase/zookeeper/HFileArchiveTracker.java https://reviews.apache.org/r/4633/#comment14537 done.
[jira] [Commented] (HBASE-5731) Make max line length 100 in linter
[ https://issues.apache.org/jira/browse/HBASE-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247504#comment-13247504 ] Phabricator commented on HBASE-5731: Kannan has accepted the revision [jira] [HBASE-5731] Make max line length 100 in linter. REVISION DETAIL https://reviews.facebook.net/D2631 BRANCH linter_100chars Make max line length 100 in linter -- Key: HBASE-5731 URL: https://issues.apache.org/jira/browse/HBASE-5731 Project: HBase Issue Type: New Feature Reporter: Mikhail Bautin Assignee: Mikhail Bautin Priority: Minor Attachments: D2625.1.patch, D2631.1.patch We have switched to 100 characters per line in our Java files. Making the change in the linter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5727) secure hbase build broke because of 'HBASE-5451 Switch RPC call envelope/headers to PBs'
[ https://issues.apache.org/jira/browse/HBASE-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247507#comment-13247507 ] Andrew Purtell commented on HBASE-5727: --- @Stack, @Devaraj: Do you want to try removing the need for separate RPC engines (maybe as a follow on issue)? secure hbase build broke because of 'HBASE-5451 Switch RPC call envelope/headers to PBs' Key: HBASE-5727 URL: https://issues.apache.org/jira/browse/HBASE-5727 Project: HBase Issue Type: Bug Reporter: stack Assignee: Devaraj Das Priority: Blocker Attachments: 5727.patch If you build with the security profile -- i.e. add '-P security' on the command line -- you'll see that the secure build is broke since we messed in rpc. Assigning Deveraj to take a look. If you can't work on this now DD, just give it back to me and I'll have a go at it. Thanks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5724) Row cache of KeyValue should be cleared in readFields().
[ https://issues.apache.org/jira/browse/HBASE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247508#comment-13247508 ] Lars Hofhansl commented on HBASE-5724: -- TestFromClientSide passes locally with this patch applied (was worried that we made incorrect assumptions somewhere). Row cache of KeyValue should be cleared in readFields(). Key: HBASE-5724 URL: https://issues.apache.org/jira/browse/HBASE-5724 Project: HBase Issue Type: Bug Affects Versions: 0.92.1 Reporter: Teruyoshi Zenmyo Assignee: Teruyoshi Zenmyo Fix For: 0.94.0, 0.96.0 Attachments: HBASE-5724.txt KeyValue does not clear its row cache in reading new values (readFields()). Therefore, If a KeyValue (kv) which caches its row bytes reads another KeyValue instance, kv.getRow() returns a wrong value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
