date:20120411


[ 
https://issues.apache.org/jira/browse/HBASE-5739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251359#comment-13251359
 ] 

Hudson commented on HBASE-5739:
---

Integrated in HBase-TRUNK-security #167 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/167/])
HBASE-5739 Upgrade guava to 11.0.2 (Revision 1311942)

 Result = SUCCESS
stack : 
Files : 
* /hbase/trunk/pom.xml
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SingleSizeCache.java


 Upgrade guava to 11.0.2
 ---

 Key: HBASE-5739
 URL: https://issues.apache.org/jira/browse/HBASE-5739
 Project: HBase
  Issue Type: Improvement
  Components: build
Affects Versions: 0.96.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
 Fix For: 0.96.0

 Attachments: hbase-5739.txt


 Hadoop has upgraded to this new version of Guava. We should, too, so we don't 
 have compatibility issues running on Hadoop 2.0+

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5759) HBaseClient throws NullPointerException when EOFException should be used.


[ 
https://issues.apache.org/jira/browse/HBASE-5759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251360#comment-13251360
 ] 

Hudson commented on HBASE-5759:
---

Integrated in HBase-TRUNK-security #167 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/167/])
HBASE-5759 HBaseClient throws NullPointerException when EOFException should 
be used. (Revision 1311899)

 Result = SUCCESS
stack : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java


 HBaseClient throws NullPointerException when EOFException should be used.
 -

 Key: HBASE-5759
 URL: https://issues.apache.org/jira/browse/HBASE-5759
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Trivial
 Fix For: 0.96.0

 Attachments: hbase-5759.patch


 When a RPC data input stream is closed, protobuf doesn't raise an 
 EOFException, it returns a null RpcResponse object.
 We need to check if the response is null before trying to access it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5758) Forward port HBASE-4109 Hostname returned via reverse dns lookup contains trailing period if configured interface is not 'default'


[ 
https://issues.apache.org/jira/browse/HBASE-5758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251358#comment-13251358
 ] 

Hudson commented on HBASE-5758:
---

Integrated in HBase-TRUNK-security #167 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/167/])
HBASE-5758 Forward port HBASE-4109 Hostname returned via reverse dns 
lookup contains trailing period if configured interface is not default 
(Revision 1311821)

 Result = SUCCESS
stack : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/Strings.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/HQuorumPeer.java


 Forward port HBASE-4109 Hostname returned via reverse dns lookup contains 
 trailing period if configured interface is not 'default'
 

 Key: HBASE-5758
 URL: https://issues.apache.org/jira/browse/HBASE-5758
 Project: HBase
  Issue Type: Task
Reporter: stack
Assignee: stack
 Fix For: 0.92.2, 0.94.0

 Attachments: 5758.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5760) Unit tests should write only under /target


[ 
https://issues.apache.org/jira/browse/HBASE-5760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251361#comment-13251361
 ] 

Hudson commented on HBASE-5760:
---

Integrated in HBase-TRUNK-security #167 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/167/])
HBASE-5760 Unit tests should write only under /target (Revision 1312043)

 Result = SUCCESS
stack : 
Files : 
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverStacking.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java


 Unit tests should write only under /target
 --

 Key: HBASE-5760
 URL: https://issues.apache.org/jira/browse/HBASE-5760
 Project: HBase
  Issue Type: Improvement
  Components: test
Affects Versions: 0.96.0
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Priority: Minor
 Fix For: 0.96.0

 Attachments: HBASE-5760_v1.patch


 Some of the unit test runs result in files under $hbase_home/test, 
 $hbase_home/build, or $hbase_home/. We should ensure that all tests use 
 target as their data location.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4109) Hostname returned via reverse dns lookup contains trailing period if configured interface is not default


[ 
https://issues.apache.org/jira/browse/HBASE-4109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251362#comment-13251362
 ] 

Hudson commented on HBASE-4109:
---

Integrated in HBase-TRUNK-security #167 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/167/])
HBASE-5758 Forward port HBASE-4109 Hostname returned via reverse dns 
lookup contains trailing period if configured interface is not default 
(Revision 1311821)

 Result = SUCCESS
stack : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/Strings.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/HQuorumPeer.java


 Hostname returned via reverse dns lookup contains trailing period if 
 configured interface is not default
 --

 Key: HBASE-4109
 URL: https://issues.apache.org/jira/browse/HBASE-4109
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Affects Versions: 0.90.3
Reporter: Shrijeet Paliwal
Assignee: Shrijeet Paliwal
 Fix For: 0.90.4

 Attachments: 
 0001-HBASE-4109-Sanitize-hostname-returned-from-DNS-class.patch


 If you are using an interface anything other than 'default' (literally that 
 keyword) DNS.java 's getDefaultHost will return a string which will 
 have a trailing period at the end. It seems javadoc of reverseDns in DNS.java 
 (see below) is conflicting with what that function is actually doing. 
 It is returning a PTR record while claims it returns a hostname. The PTR 
 record always has period at the end , RFC:  
 http://irbs.net/bog-4.9.5/bog47.html 
 We make call to DNS.getDefaultHost at more than one places and treat that as 
 actual hostname.
 Quoting HRegionServer for example
 {code}
 String machineName = DNS.getDefaultHost(conf.get(
 hbase.regionserver.dns.interface, default), conf.get(
 hbase.regionserver.dns.nameserver, default));
 {code}
 This causes inconsistencies. An example of such inconsistency was observed 
 while debugging the issue Regions not getting reassigned if RS is brought 
 down. More here 
 http://search-hadoop.com/m/CANUA1qRCkQ1 
 We may want to sanitize the string returned from DNS class. Or better we can 
 take a path of overhauling the way we do DNS name matching all over.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5755) Region sever looking for master forever with cached stale data.


[ 
https://issues.apache.org/jira/browse/HBASE-5755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251364#comment-13251364
 ] 

Hudson commented on HBASE-5755:
---

Integrated in HBase-TRUNK-security #167 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/167/])
HBASE-5755 Region sever looking for master forever with cached stale data 
(Revision 1311910)

 Result = SUCCESS
stack : 
Files : 
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterAddressTracker.java


 Region sever looking for master forever with cached stale data.
 ---

 Key: HBASE-5755
 URL: https://issues.apache.org/jira/browse/HBASE-5755
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.96.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.96.0

 Attachments: hbase-5755.patch


 When the master address tracker doesn't have the master address ZK data, or 
 the cached data is wrong, region server should not use the cached data.
 It should pull the data from ZK directly again.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5747) Forward port hbase-5708 [89-fb] Make MiniMapRedCluster directory a subdirectory of target/test

[
https://issues.apache.org/jira/browse/HBASE-5747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251371#comment-13251371
]

Hadoop QA commented on HBASE-5747:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522211/5474v2.txt
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 58 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.coprocessor.TestClassLoading
org.apache.hadoop.hbase.regionserver.wal.TestHLog
org.apache.hadoop.hbase.TestHBaseTestingUtility
org.apache.hadoop.hbase.TestFullLogReconstruction

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1473//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1473//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1473//console

This message is automatically generated.

Forward port hbase-5708 [89-fb] Make MiniMapRedCluster directory a
subdirectory of target/test

Key: HBASE-5747
URL: https://issues.apache.org/jira/browse/HBASE-5747
Project: HBase
Issue Type: Task
Reporter: stack
Assignee: stack
Priority: Blocker
Attachments: 5474.txt, 5474v2.txt

Forward port as much as we can of Mikhail's hard-won test cleanups over on
0.89 branch Will improve our being able to run unit tests in //. He also
found a few bugs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5599) [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies

2012-04-11 Thread fulin wang (Updated) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

fulin wang updated HBASE-5599:
--

Attachment: hbase-5599-trunk_v8.patch
hbase-5599-0.90_v8

Thanks for carefully review.
I have modified the 'falut'.
The patch re-upload and select ASF item.

[hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies

Key: HBASE-5599
URL: https://issues.apache.org/jira/browse/HBASE-5599
Project: HBase
Issue Type: New Feature
Components: hbck
Affects Versions: 0.90.6
Reporter: fulin wang
Assignee: fulin wang
Attachments: 0.90-surefire-report-hbck.html, hbase-5599-0.90.patch,
hbase-5599-0.90_v2.patch, hbase-5599-0.90_v3.patch, hbase-5599-0.90_v5.patch,
hbase-5599-0.90_v6.patch, hbase-5599-0.90_v7.patch, hbase-5599-0.90_v8,
hbase-5599-0.92_v5.patch, hbase-5599-0.94_v5.patch,
hbase-5599-trunk_v5.patch, hbase-5599-trunk_v7.patch,
hbase-5599-trunk_v8.patch, license.png

The hbck tool can not fix the six scenarios.
1. Version file does not exist in root dir.
Fix: I try to create a version file by 'FSUtils.setVersion' method.

2. [REGIONNAME][KEY] on HDFS, but not listed in META or deployed on any
region server.
Fix: I get region info form the hdfs file, this region info write to
'.META.' table.

3. [REGIONNAME][KEY] not in META, but deployed on [SERVERNAME]
Fix: I get region info form the hdfs file, this region info write to
'.META.' table.

4. [REGIONNAME] should not be deployed according to META, but is deployed on
[SERVERNAME]
Fix: Close this region.

5. First region should start with an empty key. You need to create a new
region and regioninfo in HDFS to plug the hole.
Fix: The region info is not in hdfs and .META., so it create a empty
region for this error.
6. There is a hole in the region chain between [KEY] and [KEY]. You need to
create a new regioninfo and region dir in hdfs to plug the hole.
Fix: The region info is not in hdfs and .META., so it create a empty region
for this hole.

[jira] [Commented] (HBASE-4676) Prefix Compression - Trie data block encoding

2012-04-11 Thread Matt Corgan (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251375#comment-13251375
 ] 

Matt Corgan commented on HBASE-4676:


Sorry, the packaging is unusual for HBase but I think it's worth a shot at 
turning this into a module rather than dumping 200+ new .java files into 
hbase-core.  As it stands, the dependencies are very limited.  The 
hbase-prefix-trie project needs to see KeyValue.java and DataBlockEncoder.java 
(maybe a couple others, but not many).  Then, hbase-core instantiates a 
PrefixTrieDataBlockEncoder through reflection at the bottom of 
DataBlockEncoding.java, so hbase-core has no compile-time dependency on 
hbase-prefix-trie.

If HBASE-4336 makes headway, then we can extract some of the fundamental hbase 
classes (KeyValue, DataBlockEncoder) into hbase-common, and I can further limit 
the prefix-trie to only have a dependency on these fundamental interfaces in 
the lightweight hbase-common module.  HBase-common will tie together the 
modules.  Limiting the dependencies between modules is the key to keeping 
development agile, otherwise small changes will trigger massive refactorings 
that even the most seasoned devs won't be able to pull off safely.

I added instructions to the git project homepage for checking out the prefix 
trie and modifying it or using it for profiling.  Just a few extra steps beyond 
applying a patch: https://github.com/hotpads/hbase-prefix-trie.  Pasting them 
here as well:

To use with Eclipse:

* check out hbase-0.94 from apache subversion to myworkspace
* apply the HBASE-4676-0.94-v1.patch from HBASE-4676

* cd myworkspace
* git clone -b 0.1 g...@github.com:hotpads/hbase-prefix-trie.git 
hbase-prefix-trie-0.1
* in eclipse - new java project
* type hbase-prefix-trie-0.1 and eclipse should find it.  continue
* on the Projects tab, add hbase-0.94.  finish
* ctrl-shift-R to open SeekBenchmarkMain.java
* follow instructions in SeekBenchmarkMain

To build a jar after modifying code:
* edit package-hbase-prefix-trie.xml in the project root to point to your build 
directory
* right click it - Run As - Ant Build

 Prefix Compression - Trie data block encoding
 -

 Key: HBASE-4676
 URL: https://issues.apache.org/jira/browse/HBASE-4676
 Project: HBase
  Issue Type: New Feature
  Components: io, performance, regionserver
Affects Versions: 0.90.6
Reporter: Matt Corgan
 Attachments: HBASE-4676-0.94-v1.patch, PrefixTrie_Format_v1.pdf, 
 PrefixTrie_Performance_v1.pdf, SeeksPerSec by blockSize.png, 
 hbase-prefix-trie-0.1.jar


 The HBase data block format has room for 2 significant improvements for 
 applications that have high block cache hit ratios.  
 First, there is no prefix compression, and the current KeyValue format is 
 somewhat metadata heavy, so there can be tremendous memory bloat for many 
 common data layouts, specifically those with long keys and short values.
 Second, there is no random access to KeyValues inside data blocks.  This 
 means that every time you double the datablock size, average seek time (or 
 average cpu consumption) goes up by a factor of 2.  The standard 64KB block 
 size is ~10x slower for random seeks than a 4KB block size, but block sizes 
 as small as 4KB cause problems elsewhere.  Using block sizes of 256KB or 1MB 
 or more may be more efficient from a disk access and block-cache perspective 
 in many big-data applications, but doing so is infeasible from a random seek 
 perspective.
 The PrefixTrie block encoding format attempts to solve both of these 
 problems.  Some features:
 * trie format for row key encoding completely eliminates duplicate row keys 
 and encodes similar row keys into a standard trie structure which also saves 
 a lot of space
 * the column family is currently stored once at the beginning of each block.  
 this could easily be modified to allow multiple family names per block
 * all qualifiers in the block are stored in their own trie format which 
 caters nicely to wide rows.  duplicate qualifers between rows are eliminated. 
  the size of this trie determines the width of the block's qualifier 
 fixed-width-int
 * the minimum timestamp is stored at the beginning of the block, and deltas 
 are calculated from that.  the maximum delta determines the width of the 
 block's timestamp fixed-width-int
 The block is structured with metadata at the beginning, then a section for 
 the row trie, then the column trie, then the timestamp deltas, and then then 
 all the values.  Most work is done in the row trie, where every leaf node 
 (corresponding to a row) contains a list of offsets/references corresponding 
 to the cells in that row.  Each cell is fixed-width to enable binary 
 searching and is represented by [1 byte operationType, X bytes qualifier

[jira] [Commented] (HBASE-5756) we can change defalult File Appender to RFA instead of DRFA.

2012-04-11 Thread rohithsharma (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251378#comment-13251378
 ] 

rohithsharma commented on HBASE-5756:
-

@stack
Thank you stack:-)  I will give patch that includes RFA properties for this for 
90,92 and 94 branches. I will make an entry to reference guide too.

What about the default behavior,is RFA or DRFA.?

 we can change defalult File Appender to RFA instead of DRFA.
 

 Key: HBASE-5756
 URL: https://issues.apache.org/jira/browse/HBASE-5756
 Project: HBase
  Issue Type: Bug
Reporter: rohithsharma
Priority: Minor

 This can be a point of concern when on a certain day the logging happens more 
 because of more and more activity. In that case the log file for that day can 
 grow huge. These logs can not be opened for analysis since size is more.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5666) RegionServer doesn't retry to check if base node is available

[
https://issues.apache.org/jira/browse/HBASE-5666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251377#comment-13251377
]

Hadoop QA commented on HBASE-5666:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522213/HBASE-5666-v8.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestSplitLogManager
org.apache.hadoop.hbase.client.TestFromClientSide

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1475//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1475//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1475//console

This message is automatically generated.

RegionServer doesn't retry to check if base node is available
-

Key: HBASE-5666
URL: https://issues.apache.org/jira/browse/HBASE-5666
Project: HBase
Issue Type: Bug
Components: regionserver, zookeeper
Affects Versions: 0.92.1, 0.94.0, 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Attachments: HBASE-5666-0.92.patch, HBASE-5666-v1.patch,
HBASE-5666-v2.patch, HBASE-5666-v3.patch, HBASE-5666-v4.patch,
HBASE-5666-v5.patch, HBASE-5666-v6.patch, HBASE-5666-v7.patch,
HBASE-5666-v8.patch, hbase-1-regionserver.log, hbase-2-regionserver.log,
hbase-3-regionserver.log, hbase-master.log, hbase-regionserver.log,
hbase-zookeeper.log

I've a script that starts hbase and a couple of region servers in distributed
mode (hbase.cluster.distributed = true)
{code}
$HBASE_HOME/bin/start-hbase.sh
$HBASE_HOME/bin/local-regionservers.sh start 1 2 3
{code}
but the region servers are not able to start...
It seems that during the RS start the the znode is still not available, and
HRegionServer.initializeZooKeeper() check just once if the base not is
available.
{code}
2012-03-28 21:54:05,013 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Check the value
configured in 'zookeeper.znode.parent'. There could be a mismatch with the
one configured in the master.
2012-03-28 21:54:08,598 FATAL
org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
localhost,60202,133296824: Initialization of RS failed. Hence aborting
RS.
java.io.IOException: Received the shutdown message while waiting.
at
org.apache.hadoop.hbase.regionserver.HRegionServer.blockAndCheckIfStopped(HRegionServer.java:626)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:596)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:558)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:672)
at java.lang.Thread.run(Thread.java:662)
{code}

[jira] [Commented] (HBASE-5677) The master never does balance because duplicate openhandled the one region

2012-04-11 Thread xufeng (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251379#comment-13251379
 ] 

xufeng commented on HBASE-5677:
---

@Lars
This issue cased by client.I think that it is not similar to HBASE-5615 in 0.90 
at least.

 The master never does balance because duplicate openhandled the one region
 --

 Key: HBASE-5677
 URL: https://issues.apache.org/jira/browse/HBASE-5677
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: HBASE-5677-90-v1.patch, 
 surefire-report_no_patched_v1.html, surefire-report_patched_v1.html


 If region be assigned When the master is doing initialization(before do 
 processFailover),the region will be duplicate openhandled.
 because the unassigned node in zookeeper will be handled again in 
 AssignmentManager#processFailover()
 it cause the region in RIT,thus the master never does balance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5599) [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies

[
https://issues.apache.org/jira/browse/HBASE-5599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251392#comment-13251392
]

Hadoop QA commented on HBASE-5599:
--

-1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12522215/hbase-5599-trunk_v8.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 6 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestSplitLogManager

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1476//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1476//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1476//console

This message is automatically generated.

[hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies

The hbck tool can not fix the six scenarios.
1. Version file does not exist in root dir.
Fix: I try to create a version file by 'FSUtils.setVersion' method.

2. [REGIONNAME][KEY] on HDFS, but not listed in META or deployed on any
region server.
Fix: I get region info form the hdfs file, this region info write to
'.META.' table.

3. [REGIONNAME][KEY] not in META, but deployed on [SERVERNAME]
Fix: I get region info form the hdfs file, this region info write to
'.META.' table.

4. [REGIONNAME] should not be deployed according to META, but is deployed on
[SERVERNAME]
Fix: Close this region.

[jira] [Updated] (HBASE-5666) RegionServer doesn't retry to check if base node is available

2012-04-11 Thread Matteo Bertozzi (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-5666:
---

Attachment: HBASE-5666-v8.patch

 RegionServer doesn't retry to check if base node is available
 -

 Key: HBASE-5666
 URL: https://issues.apache.org/jira/browse/HBASE-5666
 Project: HBase
  Issue Type: Bug
  Components: regionserver, zookeeper
Affects Versions: 0.92.1, 0.94.0, 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Attachments: HBASE-5666-0.92.patch, HBASE-5666-v1.patch, 
 HBASE-5666-v2.patch, HBASE-5666-v3.patch, HBASE-5666-v4.patch, 
 HBASE-5666-v5.patch, HBASE-5666-v6.patch, HBASE-5666-v7.patch, 
 HBASE-5666-v8.patch, hbase-1-regionserver.log, hbase-2-regionserver.log, 
 hbase-3-regionserver.log, hbase-master.log, hbase-regionserver.log, 
 hbase-zookeeper.log


 I've a script that starts hbase and a couple of region servers in distributed 
 mode (hbase.cluster.distributed = true)
 {code}
 $HBASE_HOME/bin/start-hbase.sh
 $HBASE_HOME/bin/local-regionservers.sh start 1 2 3
 {code}
 but the region servers are not able to start...
 It seems that during the RS start the the znode is still not available, and 
 HRegionServer.initializeZooKeeper() check just once if the base not is 
 available.
 {code}
 2012-03-28 21:54:05,013 INFO 
 org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Check the value 
 configured in 'zookeeper.znode.parent'. There could be a mismatch with the 
 one configured in the master.
 2012-03-28 21:54:08,598 FATAL 
 org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server 
 localhost,60202,133296824: Initialization of RS failed.  Hence aborting 
 RS.
 java.io.IOException: Received the shutdown message while waiting.
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.blockAndCheckIfStopped(HRegionServer.java:626)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:596)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:558)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:672)
   at java.lang.Thread.run(Thread.java:662)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5666) RegionServer doesn't retry to check if base node is available

2012-04-11 Thread Matteo Bertozzi (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-5666:
---

Attachment: (was: HBASE-5666-v8.patch)

 RegionServer doesn't retry to check if base node is available
 -

 Key: HBASE-5666
 URL: https://issues.apache.org/jira/browse/HBASE-5666
 Project: HBase
  Issue Type: Bug
  Components: regionserver, zookeeper
Affects Versions: 0.92.1, 0.94.0, 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Attachments: HBASE-5666-0.92.patch, HBASE-5666-v1.patch, 
 HBASE-5666-v2.patch, HBASE-5666-v3.patch, HBASE-5666-v4.patch, 
 HBASE-5666-v5.patch, HBASE-5666-v6.patch, HBASE-5666-v7.patch, 
 HBASE-5666-v8.patch, hbase-1-regionserver.log, hbase-2-regionserver.log, 
 hbase-3-regionserver.log, hbase-master.log, hbase-regionserver.log, 
 hbase-zookeeper.log


 I've a script that starts hbase and a couple of region servers in distributed 
 mode (hbase.cluster.distributed = true)
 {code}
 $HBASE_HOME/bin/start-hbase.sh
 $HBASE_HOME/bin/local-regionservers.sh start 1 2 3
 {code}
 but the region servers are not able to start...
 It seems that during the RS start the the znode is still not available, and 
 HRegionServer.initializeZooKeeper() check just once if the base not is 
 available.
 {code}
 2012-03-28 21:54:05,013 INFO 
 org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Check the value 
 configured in 'zookeeper.znode.parent'. There could be a mismatch with the 
 one configured in the master.
 2012-03-28 21:54:08,598 FATAL 
 org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server 
 localhost,60202,133296824: Initialization of RS failed.  Hence aborting 
 RS.
 java.io.IOException: Received the shutdown message while waiting.
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.blockAndCheckIfStopped(HRegionServer.java:626)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:596)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:558)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:672)
   at java.lang.Thread.run(Thread.java:662)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5757) TableInputFormat should handle as much errors as possible

2012-04-11 Thread Jan Lukavsky (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251460#comment-13251460
]

Jan Lukavsky commented on HBASE-5757:
-

The problem with multiple fetching of rows doesn't exist. I thought (don't know
why) that ScannerTimeoutException can be thrown while processing rows cached in
the scanner on client side. This is not the case. Adding counter for the number
of retries in the input format might be interesting nevertheless.

TableInputFormat should handle as much errors as possible
-

Key: HBASE-5757
URL: https://issues.apache.org/jira/browse/HBASE-5757
Project: HBase
Issue Type: Bug
Components: mapred, mapreduce
Affects Versions: 0.90.6
Reporter: Jan Lukavsky

Prior to HBASE-4196 there was different handling of IOExceptions thrown from
scanner in mapred and mapreduce API. The patch to HBASE-4196 unified this
handling so that if exception is caught a reconnect is attempted (without
bothering the mapred client). After that, HBASE-4269 changed this behavior
back, but in both mapred and mapreduce APIs. The question is, is there any
reason not to handle all errors that the input format can handle? In other
words, why not try to reissue the request after *any* IOException? I see the
following disadvantages of current approach
* the client may see exceptions like LeaseException and
ScannerTimeoutException if he fails to process all fetched data in timeout
* to avoid ScannerTimeoutException the client must raise
hbase.regionserver.lease.period
* timeouts for tasks is aready configured in mapred.task.timeout, so this
seems to me a bit redundant, because typically one needs to update both these
parameters
* I don't see any possibility to get rid of LeaseException (this is
configured on server side)
I think all of these issues would be gone, if the DoNotRetryIOException would
not be rethrown. On the other hand, handling errors in InputFormat has
disadvantage, that it may hide from the user some inefficiency. Eg. if I have
very big scanner.caching, and I manage to process only a few rows in timeout,
I will end up with single row being fetched many times (and will not be
explicitly notified about this). Could we solve this problem by adding some
counter to the InputFormat?

[jira] [Updated] (HBASE-5757) TableInputFormat should handle as much errors as possible

2012-04-11 Thread Jan Lukavsky (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Lukavsky updated HBASE-5757:


Description: 
Prior to HBASE-4196 there was different handling of IOExceptions thrown from 
scanner in mapred and mapreduce API. The patch to HBASE-4196 unified this 
handling so that if exception is caught a reconnect is attempted (without 
bothering the mapred client). After that, HBASE-4269 changed this behavior 
back, but in both mapred and mapreduce APIs. The question is, is there any 
reason not to handle all errors that the input format can handle? In other 
words, why not try to reissue the request after *any* IOException? I see the 
following disadvantages of current approach
 * the client may see exceptions like LeaseException and 
ScannerTimeoutException if he fails to process all fetched data in timeout
 * to avoid ScannerTimeoutException the client must raise 
hbase.regionserver.lease.period
 * timeouts for tasks is aready configured in mapred.task.timeout, so this 
seems to me a bit redundant, because typically one needs to update both these 
parameters
 * I don't see any possibility to get rid of LeaseException (this is configured 
on server side)

I think all of these issues would be gone, if the DoNotRetryIOException would 
not be rethrown. -On the other hand, handling errors in InputFormat has 
disadvantage, that it may hide from the user some inefficiency. Eg. if I have 
very big scanner.caching, and I manage to process only a few rows in timeout, I 
will end up with single row being fetched many times (and will not be 
explicitly notified about this). Could we solve this problem by adding some 
counter to the InputFormat?-

  was:
Prior to HBASE-4196 there was different handling of IOExceptions thrown from 
scanner in mapred and mapreduce API. The patch to HBASE-4196 unified this 
handling so that if exception is caught a reconnect is attempted (without 
bothering the mapred client). After that, HBASE-4269 changed this behavior 
back, but in both mapred and mapreduce APIs. The question is, is there any 
reason not to handle all errors that the input format can handle? In other 
words, why not try to reissue the request after *any* IOException? I see the 
following disadvantages of current approach
 * the client may see exceptions like LeaseException and 
ScannerTimeoutException if he fails to process all fetched data in timeout
 * to avoid ScannerTimeoutException the client must raise 
hbase.regionserver.lease.period
 * timeouts for tasks is aready configured in mapred.task.timeout, so this 
seems to me a bit redundant, because typically one needs to update both these 
parameters
 * I don't see any possibility to get rid of LeaseException (this is configured 
on server side)

I think all of these issues would be gone, if the DoNotRetryIOException would 
not be rethrown. On the other hand, handling errors in InputFormat has 
disadvantage, that it may hide from the user some inefficiency. Eg. if I have 
very big scanner.caching, and I manage to process only a few rows in timeout, I 
will end up with single row being fetched many times (and will not be 
explicitly notified about this). Could we solve this problem by adding some 
counter to the InputFormat?


 TableInputFormat should handle as much errors as possible
 -

 Key: HBASE-5757
 URL: https://issues.apache.org/jira/browse/HBASE-5757
 Project: HBase
  Issue Type: Bug
  Components: mapred, mapreduce
Affects Versions: 0.90.6
Reporter: Jan Lukavsky

 Prior to HBASE-4196 there was different handling of IOExceptions thrown from 
 scanner in mapred and mapreduce API. The patch to HBASE-4196 unified this 
 handling so that if exception is caught a reconnect is attempted (without 
 bothering the mapred client). After that, HBASE-4269 changed this behavior 
 back, but in both mapred and mapreduce APIs. The question is, is there any 
 reason not to handle all errors that the input format can handle? In other 
 words, why not try to reissue the request after *any* IOException? I see the 
 following disadvantages of current approach
  * the client may see exceptions like LeaseException and 
 ScannerTimeoutException if he fails to process all fetched data in timeout
  * to avoid ScannerTimeoutException the client must raise 
 hbase.regionserver.lease.period
  * timeouts for tasks is aready configured in mapred.task.timeout, so this 
 seems to me a bit redundant, because typically one needs to update both these 
 parameters
  * I don't see any possibility to get rid of LeaseException (this is 
 configured on server side)
 I think all of these issues would be gone, if the DoNotRetryIOException would 
 not be rethrown. -On the other hand, handling errors in InputFormat has 
 disadvantage, that it may hide from the user

[jira] [Commented] (HBASE-5666) RegionServer doesn't retry to check if base node is available

[
https://issues.apache.org/jira/browse/HBASE-5666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251462#comment-13251462
]

Hadoop QA commented on HBASE-5666:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/1250/HBASE-5666-v8.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.wal.TestHLog

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1477//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1477//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1477//console

This message is automatically generated.

RegionServer doesn't retry to check if base node is available
-

[jira] [Created] (HBASE-5764) [book] adding entry in Config chapter about turning spec-ex off by default

2012-04-11 Thread Doug Meil (Created) (JIRA)

[book] adding entry in Config chapter about turning spec-ex off by default
--

 Key: HBASE-5764
 URL: https://issues.apache.org/jira/browse/HBASE-5764
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor


configuration.xml
* adding entry in recommended configs about turning speculative execution 
off.  This has come up enough times on the dist-list where it needs to be cited 
in the Config chapter.  It was already in the MapReduce chapter, but I am 
adding to to config (and having the MR chapter link to the new section)

book.xml
* Adding xref to MapReduce chapter to new specex config entry in Config chapter.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5764) [book] adding entry in Config chapter about turning spec-ex off by default


 [ 
https://issues.apache.org/jira/browse/HBASE-5764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-5764:
-

Status: Patch Available  (was: Open)

 [book] adding entry in Config chapter about turning spec-ex off by default
 --

 Key: HBASE-5764
 URL: https://issues.apache.org/jira/browse/HBASE-5764
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: docbkx_hbase_5764.patch


 configuration.xml
 * adding entry in recommended configs about turning speculative execution 
 off.  This has come up enough times on the dist-list where it needs to be 
 cited in the Config chapter.  It was already in the MapReduce chapter, but I 
 am adding to to config (and having the MR chapter link to the new section)
 book.xml
 * Adding xref to MapReduce chapter to new specex config entry in Config 
 chapter.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5764) [book] adding entry in Config chapter about turning spec-ex off by default


 [ 
https://issues.apache.org/jira/browse/HBASE-5764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-5764:
-

Attachment: docbkx_hbase_5764.patch

 [book] adding entry in Config chapter about turning spec-ex off by default
 --

 Key: HBASE-5764
 URL: https://issues.apache.org/jira/browse/HBASE-5764
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: docbkx_hbase_5764.patch


 configuration.xml
 * adding entry in recommended configs about turning speculative execution 
 off.  This has come up enough times on the dist-list where it needs to be 
 cited in the Config chapter.  It was already in the MapReduce chapter, but I 
 am adding to to config (and having the MR chapter link to the new section)
 book.xml
 * Adding xref to MapReduce chapter to new specex config entry in Config 
 chapter.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5764) [book] adding entry in Config chapter about turning spec-ex off by default


 [ 
https://issues.apache.org/jira/browse/HBASE-5764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-5764:
-

Resolution: Fixed
Status: Resolved  (was: Patch Available)

 [book] adding entry in Config chapter about turning spec-ex off by default
 --

 Key: HBASE-5764
 URL: https://issues.apache.org/jira/browse/HBASE-5764
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: docbkx_hbase_5764.patch


 configuration.xml
 * adding entry in recommended configs about turning speculative execution 
 off.  This has come up enough times on the dist-list where it needs to be 
 cited in the Config chapter.  It was already in the MapReduce chapter, but I 
 am adding to to config (and having the MR chapter link to the new section)
 book.xml
 * Adding xref to MapReduce chapter to new specex config entry in Config 
 chapter.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5765) [book] adding ImportTsv to Ops chapter

2012-04-11 Thread Doug Meil (Created) (JIRA)

[book] adding ImportTsv to Ops chapter
--

 Key: HBASE-5765
 URL: https://issues.apache.org/jira/browse/HBASE-5765
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor


ops_mgt.xml
* adding ImportTsv to Ops chapter.  Import was already in there, but not 
ImportTsv

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5765) [book] adding ImportTsv to Ops chapter


 [ 
https://issues.apache.org/jira/browse/HBASE-5765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-5765:
-

Attachment: ops_mgt_hbase_5765.xml.patch

 [book] adding ImportTsv to Ops chapter
 --

 Key: HBASE-5765
 URL: https://issues.apache.org/jira/browse/HBASE-5765
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: ops_mgt_hbase_5765.xml.patch


 ops_mgt.xml
 * adding ImportTsv to Ops chapter.  Import was already in there, but not 
 ImportTsv

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5765) [book] adding ImportTsv to Ops chapter


 [ 
https://issues.apache.org/jira/browse/HBASE-5765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-5765:
-

Resolution: Fixed
Status: Resolved  (was: Patch Available)

 [book] adding ImportTsv to Ops chapter
 --

 Key: HBASE-5765
 URL: https://issues.apache.org/jira/browse/HBASE-5765
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: ops_mgt_hbase_5765.xml.patch


 ops_mgt.xml
 * adding ImportTsv to Ops chapter.  Import was already in there, but not 
 ImportTsv

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5765) [book] adding ImportTsv to Ops chapter


 [ 
https://issues.apache.org/jira/browse/HBASE-5765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-5765:
-

Status: Patch Available  (was: Open)

 [book] adding ImportTsv to Ops chapter
 --

 Key: HBASE-5765
 URL: https://issues.apache.org/jira/browse/HBASE-5765
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: ops_mgt_hbase_5765.xml.patch


 ops_mgt.xml
 * adding ImportTsv to Ops chapter.  Import was already in there, but not 
 ImportTsv

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5766) [book] Add 'bulk loading' to Ops chapter utilities

2012-04-11 Thread Doug Meil (Created) (JIRA)

[book] Add 'bulk loading' to Ops chapter utilities
--

 Key: HBASE-5766
 URL: https://issues.apache.org/jira/browse/HBASE-5766
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: ops_mgt_hbase_5766.xml.patch

ops_mgt.xml
* adding entry under tools for Bulk Loading.  This section points to the 
external bulk loads web-page which existed before the RefGuide was created, 
but it's still the best source of info on this topic.
* That external web-page needs to be merged in to the RefGuide and it's on my 
to-do list.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5766) [book] Add 'bulk loading' to Ops chapter utilities


 [ 
https://issues.apache.org/jira/browse/HBASE-5766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-5766:
-

Attachment: ops_mgt_hbase_5766.xml.patch

 [book] Add 'bulk loading' to Ops chapter utilities
 --

 Key: HBASE-5766
 URL: https://issues.apache.org/jira/browse/HBASE-5766
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: ops_mgt_hbase_5766.xml.patch


 ops_mgt.xml
 * adding entry under tools for Bulk Loading.  This section points to the 
 external bulk loads web-page which existed before the RefGuide was created, 
 but it's still the best source of info on this topic.
 * That external web-page needs to be merged in to the RefGuide and it's on my 
 to-do list.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5766) [book] Add 'bulk loading' to Ops chapter utilities


 [ 
https://issues.apache.org/jira/browse/HBASE-5766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-5766:
-

Status: Patch Available  (was: Open)

 [book] Add 'bulk loading' to Ops chapter utilities
 --

 Key: HBASE-5766
 URL: https://issues.apache.org/jira/browse/HBASE-5766
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: ops_mgt_hbase_5766.xml.patch


 ops_mgt.xml
 * adding entry under tools for Bulk Loading.  This section points to the 
 external bulk loads web-page which existed before the RefGuide was created, 
 but it's still the best source of info on this topic.
 * That external web-page needs to be merged in to the RefGuide and it's on my 
 to-do list.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5766) [book] Add 'bulk loading' to Ops chapter utilities


 [ 
https://issues.apache.org/jira/browse/HBASE-5766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-5766:
-

Resolution: Fixed
Status: Resolved  (was: Patch Available)

 [book] Add 'bulk loading' to Ops chapter utilities
 --

 Key: HBASE-5766
 URL: https://issues.apache.org/jira/browse/HBASE-5766
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: ops_mgt_hbase_5766.xml.patch


 ops_mgt.xml
 * adding entry under tools for Bulk Loading.  This section points to the 
 external bulk loads web-page which existed before the RefGuide was created, 
 but it's still the best source of info on this topic.
 * That external web-page needs to be merged in to the RefGuide and it's on my 
 to-do list.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5767) Add the hbase shell table_att for any attribute

2012-04-11 Thread Xing Shi (Created) (JIRA)

Add the hbase shell table_att for any attribute
---

 Key: HBASE-5767
 URL: https://issues.apache.org/jira/browse/HBASE-5767
 Project: HBase
  Issue Type: Improvement
  Components: shell
Reporter: Xing Shi
Priority: Minor


Now the HTableDescriptor supports setValue(String key, String value) method, 
but the hbase shell not support it.

May be like this:
{quota}
hbase(main):003:0 alter 'test', METHOD='table_att', 'key1'='value1'
Updating all regions with the new schema...
1/1 regions updated.
Done.
0 row(s) in 1.0820 seconds

hbase(main):005:0 describe 'test'
DESCRIPTION 
  ENABLED  
 {NAME = 'test', key1 = 'value1', FAMILIES = [{NAME = 'f1', BLOOMFILTER = 
'NONE', RE true 
 PLICATION_SCOPE = '0', VERSIONS = '3', COMPRESSION = 'NONE', MIN_VERSIONS 
= '0', TTL  
  = '2147483647', BLOCKSIZE = '65536', IN_MEMORY = 'false', BLOCKCACHE = 
'true'}]} 
1 row(s) in 0.0300 seconds

hbase(main):007:0 alter 'test', METHOD='table_att_unset', NAME='key1'
Updating all regions with the new schema...
1/1 regions updated.
Done.
0 row(s) in 1.0860 seconds

hbase(main):008:0 describe 'test'
DESCRIPTION 
  ENABLED  
 {NAME = 'test', FAMILIES = [{NAME = 'f1', BLOOMFILTER = 'NONE', 
REPLICATION_SCOPE = false
  '0', VERSIONS = '3', COMPRESSION = 'NONE', MIN_VERSIONS = '0', TTL = 
'2147483647',   
 BLOCKSIZE = '65536', IN_MEMORY = 'false', BLOCKCACHE = 'true'}]}
   
1 row(s) in 0.0280 seconds
{quota}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5767) Add the hbase shell table_att for any attribute

2012-04-11 Thread ShiXing (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ShiXing updated HBASE-5767:
---

Attachment: HBASE-5767.patch

att

 Add the hbase shell table_att for any attribute
 ---

 Key: HBASE-5767
 URL: https://issues.apache.org/jira/browse/HBASE-5767
 Project: HBase
  Issue Type: Improvement
  Components: shell
Reporter: Xing Shi
Priority: Minor
 Attachments: HBASE-5767.patch


 Now the HTableDescriptor supports setValue(String key, String value) method, 
 but the hbase shell not support it.
 May be like this:
 {quota}
 hbase(main):003:0 alter 'test', METHOD='table_att', 'key1'='value1'
 Updating all regions with the new schema...
 1/1 regions updated.
 Done.
 0 row(s) in 1.0820 seconds
 hbase(main):005:0 describe 'test'
 DESCRIPTION   
 ENABLED  
  {NAME = 'test', key1 = 'value1', FAMILIES = [{NAME = 'f1', BLOOMFILTER 
 = 'NONE', RE true 
  PLICATION_SCOPE = '0', VERSIONS = '3', COMPRESSION = 'NONE', MIN_VERSIONS 
 = '0', TTL  
   = '2147483647', BLOCKSIZE = '65536', IN_MEMORY = 'false', BLOCKCACHE = 
 'true'}]} 
 1 row(s) in 0.0300 seconds
 hbase(main):007:0 alter 'test', METHOD='table_att_unset', NAME='key1'
 Updating all regions with the new schema...
 1/1 regions updated.
 Done.
 0 row(s) in 1.0860 seconds
 hbase(main):008:0 describe 'test'
 DESCRIPTION   
 ENABLED  
  {NAME = 'test', FAMILIES = [{NAME = 'f1', BLOOMFILTER = 'NONE', 
 REPLICATION_SCOPE = false
   '0', VERSIONS = '3', COMPRESSION = 'NONE', MIN_VERSIONS = '0', TTL = 
 '2147483647',   
  BLOCKSIZE = '65536', IN_MEMORY = 'false', BLOCKCACHE = 'true'}]}  
  
 1 row(s) in 0.0280 seconds
 {quota}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5765) [book] adding ImportTsv to Ops chapter


[ 
https://issues.apache.org/jira/browse/HBASE-5765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251565#comment-13251565
 ] 

Hudson commented on HBASE-5765:
---

Integrated in HBase-TRUNK #2742 (See 
[https://builds.apache.org/job/HBase-TRUNK/2742/])
hbase-5765 adding ImportTsv to Ops chapter (Revision 1324736)

 Result = SUCCESS

 [book] adding ImportTsv to Ops chapter
 --

 Key: HBASE-5765
 URL: https://issues.apache.org/jira/browse/HBASE-5765
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: ops_mgt_hbase_5765.xml.patch


 ops_mgt.xml
 * adding ImportTsv to Ops chapter.  Import was already in there, but not 
 ImportTsv

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5764) [book] adding entry in Config chapter about turning spec-ex off by default


[ 
https://issues.apache.org/jira/browse/HBASE-5764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251564#comment-13251564
 ] 

Hudson commented on HBASE-5764:
---

Integrated in HBase-TRUNK #2742 (See 
[https://builds.apache.org/job/HBase-TRUNK/2742/])
hbase-5764 adding turning off SpecEx in Config chapter recommendations 
(Revision 1324730)

 Result = SUCCESS

 [book] adding entry in Config chapter about turning spec-ex off by default
 --

 Key: HBASE-5764
 URL: https://issues.apache.org/jira/browse/HBASE-5764
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: docbkx_hbase_5764.patch


 configuration.xml
 * adding entry in recommended configs about turning speculative execution 
 off.  This has come up enough times on the dist-list where it needs to be 
 cited in the Config chapter.  It was already in the MapReduce chapter, but I 
 am adding to to config (and having the MR chapter link to the new section)
 book.xml
 * Adding xref to MapReduce chapter to new specex config entry in Config 
 chapter.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5766) [book] Add 'bulk loading' to Ops chapter utilities


[ 
https://issues.apache.org/jira/browse/HBASE-5766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251566#comment-13251566
 ] 

Hudson commented on HBASE-5766:
---

Integrated in HBase-TRUNK #2742 (See 
[https://builds.apache.org/job/HBase-TRUNK/2742/])
hbase-5766.  adding bulk loading to Ops chapter tools list. (Revision 
1324741)

 Result = SUCCESS

 [book] Add 'bulk loading' to Ops chapter utilities
 --

 Key: HBASE-5766
 URL: https://issues.apache.org/jira/browse/HBASE-5766
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: ops_mgt_hbase_5766.xml.patch


 ops_mgt.xml
 * adding entry under tools for Bulk Loading.  This section points to the 
 external bulk loads web-page which existed before the RefGuide was created, 
 but it's still the best source of info on this topic.
 * That external web-page needs to be merged in to the RefGuide and it's on my 
 to-do list.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5767) Add the hbase shell table_att for any attribute

2012-04-11 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251650#comment-13251650
 ] 

Zhihong Yu commented on HBASE-5767:
---

@Xing:
Can you add a unit test for your patch ?

 Add the hbase shell table_att for any attribute
 ---

 Key: HBASE-5767
 URL: https://issues.apache.org/jira/browse/HBASE-5767
 Project: HBase
  Issue Type: Improvement
  Components: shell
Reporter: Xing Shi
Priority: Minor
 Attachments: HBASE-5767.patch


 Now the HTableDescriptor supports setValue(String key, String value) method, 
 but the hbase shell not support it.
 May be like this:
 {quota}
 hbase(main):003:0 alter 'test', METHOD='table_att', 'key1'='value1'
 Updating all regions with the new schema...
 1/1 regions updated.
 Done.
 0 row(s) in 1.0820 seconds
 hbase(main):005:0 describe 'test'
 DESCRIPTION   
 ENABLED  
  {NAME = 'test', key1 = 'value1', FAMILIES = [{NAME = 'f1', BLOOMFILTER 
 = 'NONE', RE true 
  PLICATION_SCOPE = '0', VERSIONS = '3', COMPRESSION = 'NONE', MIN_VERSIONS 
 = '0', TTL  
   = '2147483647', BLOCKSIZE = '65536', IN_MEMORY = 'false', BLOCKCACHE = 
 'true'}]} 
 1 row(s) in 0.0300 seconds
 hbase(main):007:0 alter 'test', METHOD='table_att_unset', NAME='key1'
 Updating all regions with the new schema...
 1/1 regions updated.
 Done.
 0 row(s) in 1.0860 seconds
 hbase(main):008:0 describe 'test'
 DESCRIPTION   
 ENABLED  
  {NAME = 'test', FAMILIES = [{NAME = 'f1', BLOOMFILTER = 'NONE', 
 REPLICATION_SCOPE = false
   '0', VERSIONS = '3', COMPRESSION = 'NONE', MIN_VERSIONS = '0', TTL = 
 '2147483647',   
  BLOCKSIZE = '65536', IN_MEMORY = 'false', BLOCKCACHE = 'true'}]}  
  
 1 row(s) in 0.0280 seconds
 {quota}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped mega regions


[ 
https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251683#comment-13251683
 ] 

jirapos...@reviews.apache.org commented on HBASE-5719:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4649/#review6851
---

Ship it!


Looks good to me Jimmy.  Mind checking 0.90/0.92/0.94 and doing ports if 
necessary?  Should be trivial.

I have one nit that you can address or ignore. :)


src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java
https://reviews.apache.org/r/4649/#comment15241

nit: would be easier to read if 
assertTrue((r1.equals(ac)  r2.equals(ae)) || (r1.equals(ae)  
r2.equals(ac)));


- jmhsieh


On 2012-04-09 19:27:53, Jimmy Xiang wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4649/
bq.  ---
bq.  
bq.  (Updated 2012-04-09 19:27:53)
bq.  
bq.  
bq.  Review request for hbase and jmhsieh.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Make it configurable to sideline some regions in big overlapped groups 
which hbck doesn't handle currently.
bq.  
bq.  The regions chose to sideline are those which overlap with most other 
regions.
bq.  
bq.  
bq.  This addresses bug HBASE-5719.
bq.  https://issues.apache.org/jira/browse/HBASE-5719
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 54f9b21 
bq.src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java 
17678dd 
bq.
src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java 
ac3b225 
bq.  
bq.  Diff: https://reviews.apache.org/r/4649/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  mvn -PlocalTests -Dtest=TestHBaseFsck* clean test
bq.  
bq.  Also tested in real system to fix inconsistencies.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Jimmy
bq.  
bq.



 Enhance hbck to sideline overlapped mega regions
 

 Key: HBASE-5719
 URL: https://issues.apache.org/jira/browse/HBASE-5719
 Project: HBase
  Issue Type: New Feature
  Components: hbck
Affects Versions: 0.94.0, 0.96.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.96.0

 Attachments: hbase-5719.patch


 If there are too many regions in one overlapped group (by default, more than 
 10), hbck currently doesn't merge them since it takes time.
 In this case, we can sideline some regions in the group and break the 
 overlapping to fix the inconsistency.  Later on, sidelined regions can be 
 bulk loaded manually.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5599) [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies

[
https://issues.apache.org/jira/browse/HBASE-5599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Hsieh updated HBASE-5599:
--

Comment: was deleted

(was: -1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522208/license.png
against trunk revision .

+1 @author. The patch does not contain any @author tags.

-1 patch. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1472//console

This message is automatically generated.)

[hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies

The hbck tool can not fix the six scenarios.
1. Version file does not exist in root dir.
Fix: I try to create a version file by 'FSUtils.setVersion' method.

2. [REGIONNAME][KEY] on HDFS, but not listed in META or deployed on any
region server.
Fix: I get region info form the hdfs file, this region info write to
'.META.' table.

3. [REGIONNAME][KEY] not in META, but deployed on [SERVERNAME]
Fix: I get region info form the hdfs file, this region info write to
'.META.' table.

4. [REGIONNAME] should not be deployed according to META, but is deployed on
[SERVERNAME]
Fix: Close this region.

[jira] [Updated] (HBASE-5767) Add the hbase shell table_att for any attribute

2012-04-11 Thread ShiXing (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ShiXing updated HBASE-5767:
---

Attachment: HBASE-5767-V2.patch

add unit test

 Add the hbase shell table_att for any attribute
 ---

 Key: HBASE-5767
 URL: https://issues.apache.org/jira/browse/HBASE-5767
 Project: HBase
  Issue Type: Improvement
  Components: shell
Reporter: Xing Shi
Priority: Minor
 Attachments: HBASE-5767-V2.patch, HBASE-5767.patch


 Now the HTableDescriptor supports setValue(String key, String value) method, 
 but the hbase shell not support it.
 May be like this:
 {quota}
 hbase(main):003:0 alter 'test', METHOD='table_att', 'key1'='value1'
 Updating all regions with the new schema...
 1/1 regions updated.
 Done.
 0 row(s) in 1.0820 seconds
 hbase(main):005:0 describe 'test'
 DESCRIPTION   
 ENABLED  
  {NAME = 'test', key1 = 'value1', FAMILIES = [{NAME = 'f1', BLOOMFILTER 
 = 'NONE', RE true 
  PLICATION_SCOPE = '0', VERSIONS = '3', COMPRESSION = 'NONE', MIN_VERSIONS 
 = '0', TTL  
   = '2147483647', BLOCKSIZE = '65536', IN_MEMORY = 'false', BLOCKCACHE = 
 'true'}]} 
 1 row(s) in 0.0300 seconds
 hbase(main):007:0 alter 'test', METHOD='table_att_unset', NAME='key1'
 Updating all regions with the new schema...
 1/1 regions updated.
 Done.
 0 row(s) in 1.0860 seconds
 hbase(main):008:0 describe 'test'
 DESCRIPTION   
 ENABLED  
  {NAME = 'test', FAMILIES = [{NAME = 'f1', BLOOMFILTER = 'NONE', 
 REPLICATION_SCOPE = false
   '0', VERSIONS = '3', COMPRESSION = 'NONE', MIN_VERSIONS = '0', TTL = 
 '2147483647',   
  BLOCKSIZE = '65536', IN_MEMORY = 'false', BLOCKCACHE = 'true'}]}  
  
 1 row(s) in 0.0280 seconds
 {quota}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5666) RegionServer doesn't retry to check if base node is available

2012-04-11 Thread Nicolas Spiegelberg (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Zhihong Yu updated HBASE-5666:
--

Comment: was deleted

(was: -1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12521401/HBASE-5666-v5.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:

org.apache.hadoop.hbase.mapreduce.TestMultithreadedTableMapper
org.apache.hadoop.hbase.client.TestInstantSchemaChangeSplit
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
org.apache.hadoop.hbase.mapreduce.TestTableMapReduce

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1395//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1395//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1395//console

This message is automatically generated.)

RegionServer doesn't retry to check if base node is available
-

[jira] [Commented] (HBASE-5767) Add the hbase shell table_att for any attribute


[ 
https://issues.apache.org/jira/browse/HBASE-5767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251696#comment-13251696
 ] 

Nicolas Spiegelberg commented on HBASE-5767:


What's the reason for giving access to the entire KV map?  Note that I just 
added HBASE-5335, which assumes that unreserved KV types are config values.  
Hopefully, that JIRA already solved your issue?

 Add the hbase shell table_att for any attribute
 ---

 Key: HBASE-5767
 URL: https://issues.apache.org/jira/browse/HBASE-5767
 Project: HBase
  Issue Type: Improvement
  Components: shell
Reporter: Xing Shi
Priority: Minor
 Attachments: HBASE-5767-V2.patch, HBASE-5767.patch


 Now the HTableDescriptor supports setValue(String key, String value) method, 
 but the hbase shell not support it.
 May be like this:
 {quota}
 hbase(main):003:0 alter 'test', METHOD='table_att', 'key1'='value1'
 Updating all regions with the new schema...
 1/1 regions updated.
 Done.
 0 row(s) in 1.0820 seconds
 hbase(main):005:0 describe 'test'
 DESCRIPTION   
 ENABLED  
  {NAME = 'test', key1 = 'value1', FAMILIES = [{NAME = 'f1', BLOOMFILTER 
 = 'NONE', RE true 
  PLICATION_SCOPE = '0', VERSIONS = '3', COMPRESSION = 'NONE', MIN_VERSIONS 
 = '0', TTL  
   = '2147483647', BLOCKSIZE = '65536', IN_MEMORY = 'false', BLOCKCACHE = 
 'true'}]} 
 1 row(s) in 0.0300 seconds
 hbase(main):007:0 alter 'test', METHOD='table_att_unset', NAME='key1'
 Updating all regions with the new schema...
 1/1 regions updated.
 Done.
 0 row(s) in 1.0860 seconds
 hbase(main):008:0 describe 'test'
 DESCRIPTION   
 ENABLED  
  {NAME = 'test', FAMILIES = [{NAME = 'f1', BLOOMFILTER = 'NONE', 
 REPLICATION_SCOPE = false
   '0', VERSIONS = '3', COMPRESSION = 'NONE', MIN_VERSIONS = '0', TTL = 
 '2147483647',   
  BLOCKSIZE = '65536', IN_MEMORY = 'false', BLOCKCACHE = 'true'}]}  
  
 1 row(s) in 0.0280 seconds
 {quota}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5767) Add the hbase shell table_att for any attribute

2012-04-11 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5767:
--

Hadoop Flags: Reviewed
  Status: Patch Available  (was: Open)

 Add the hbase shell table_att for any attribute
 ---

 Key: HBASE-5767
 URL: https://issues.apache.org/jira/browse/HBASE-5767
 Project: HBase
  Issue Type: Improvement
  Components: shell
Reporter: Xing Shi
Priority: Minor
 Attachments: HBASE-5767-V2.patch, HBASE-5767.patch


 Now the HTableDescriptor supports setValue(String key, String value) method, 
 but the hbase shell not support it.
 May be like this:
 {quota}
 hbase(main):003:0 alter 'test', METHOD='table_att', 'key1'='value1'
 Updating all regions with the new schema...
 1/1 regions updated.
 Done.
 0 row(s) in 1.0820 seconds
 hbase(main):005:0 describe 'test'
 DESCRIPTION   
 ENABLED  
  {NAME = 'test', key1 = 'value1', FAMILIES = [{NAME = 'f1', BLOOMFILTER 
 = 'NONE', RE true 
  PLICATION_SCOPE = '0', VERSIONS = '3', COMPRESSION = 'NONE', MIN_VERSIONS 
 = '0', TTL  
   = '2147483647', BLOCKSIZE = '65536', IN_MEMORY = 'false', BLOCKCACHE = 
 'true'}]} 
 1 row(s) in 0.0300 seconds
 hbase(main):007:0 alter 'test', METHOD='table_att_unset', NAME='key1'
 Updating all regions with the new schema...
 1/1 regions updated.
 Done.
 0 row(s) in 1.0860 seconds
 hbase(main):008:0 describe 'test'
 DESCRIPTION   
 ENABLED  
  {NAME = 'test', FAMILIES = [{NAME = 'f1', BLOOMFILTER = 'NONE', 
 REPLICATION_SCOPE = false
   '0', VERSIONS = '3', COMPRESSION = 'NONE', MIN_VERSIONS = '0', TTL = 
 '2147483647',   
  BLOCKSIZE = '65536', IN_MEMORY = 'false', BLOCKCACHE = 'true'}]}  
  
 1 row(s) in 0.0280 seconds
 {quota}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5620) Convert the client protocol of HRegionInterface to PB


[ 
https://issues.apache.org/jira/browse/HBASE-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251719#comment-13251719
 ] 

jirapos...@reviews.apache.org commented on HBASE-5620:
--



bq.  On 2012-04-11 06:22:10, Michael Stack wrote:
bq.   I got as far as HTable.  Still have more to go.  This is some pretty 
great work Jimmy.  This stuff is hard.  One thought I was having that is a 
little related (but it can be safely ignored) is what if there were only one 
method on an HRegionServer and you gave it an array of Gets, Puts, Delete, 
Increment, pb messages and you just let the regionserver sort it out.  is that 
even posssible?  I'll do rest of review tomorrow.

Thanks for reviewing. As to one method to handle all actions, it is supported 
by the multi call.  It is too complex to match the response.


bq.  On 2012-04-11 06:22:10, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java, line 691
bq.   https://reviews.apache.org/r/4629/diff/2/?file=100870#file100870line691
bq.  
bq.   I said it before but I like this code removal

It is not used so I removed it.


bq.  On 2012-04-11 06:22:10, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 533
bq.   https://reviews.apache.org/r/4629/diff/2/?file=100872#file100872line533
bq.  
bq.   What about the EOFE you added today?  Does that need to go in here 
anywhere?

EOFE is an IOException.  Good catch. I addressed it in HBASE-5621, which is in 
testing now.


bq.  On 2012-04-11 06:22:10, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 574
bq.   https://reviews.apache.org/r/4629/diff/2/?file=100872#file100872line574
bq.  
bq.   So, to be clear, if a client does not close a scanner now, how does 
it get cleaned up on the server?  Does it live forever?

In this case, the Leases daemon will expire the scanner and clean it up.


bq.  On 2012-04-11 06:22:10, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, 
line 184
bq.   https://reviews.apache.org/r/4629/diff/2/?file=100874#file100874line184
bq.  
bq.   Rename to hbase.clientprotocol.class and CLIENT_PROTOCOL_CLASS.  
Seems more consistent?

Will fix.


bq.  On 2012-04-11 06:22:10, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/client/HConnection.java, line 235
bq.   https://reviews.apache.org/r/4629/diff/2/?file=100873#file100873line235
bq.  
bq.   Does this feel right Jimmy?  Seems odd asking an HConnection for a 
ClientProtocol?  It feels like a ClientProtocol should have an HConnection?  Or 
that we'd put together a ClientProtocol + an HConnection for it to use in a 
different object altogether.  What you think?
bq.   
bq.   Maybe this is a question for another client implementation that we 
do later?   But would be interested if you had any ideas on this.

I agree a ClientProtocol should have an HConnection.  We can hide the 
HConnection. Given a region location, we can create a ClientProtocol which 
manages its own HConnection.


bq.  On 2012-04-11 06:22:10, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, 
line 187
bq.   https://reviews.apache.org/r/4629/diff/2/?file=100874#file100874line187
bq.  
bq.   Rename too accordingly.

Will fix.


bq.  On 2012-04-11 06:22:10, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, 
line 565
bq.   https://reviews.apache.org/r/4629/diff/2/?file=100874#file100874line565
bq.  
bq.   This is pb VersionedProtocol?

So far, it can be any VersionedProtocol: ClientProtocol or HRegionInterface.


bq.  On 2012-04-11 06:22:10, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, 
line 621
bq.   https://reviews.apache.org/r/4629/diff/2/?file=100874#file100874line621
bq.  
bq.   Want to adjust this message?

Fixed.


bq.  On 2012-04-11 06:22:10, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, 
line 1440
bq.   
https://reviews.apache.org/r/4629/diff/2/?file=100874#file100874line1440
bq.  
bq.   This set of protocols is covered by the synchronize held above? i.e. 
all access on protocols are under a block like this?

Good point.  Will fix.


bq.  On 2012-04-11 06:22:10, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, 
line 1449
bq.   
https://reviews.apache.org/r/4629/diff/2/?file=100874#file100874line1449
bq.  
bq.   Is there a difference here?  We create an ISA only when we need it 
previously?  Now we create it always?  Is there a diff here?

HServerAddress is deprecated. We are not going to get an ISA passed in.  The 
protocol is cached so we don't need to create it many times.


- Jimmy

[jira] [Commented] (HBASE-5547) Don't delete HFiles when in backup mode

2012-04-11 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251727#comment-13251727
 ] 

jirapos...@reviews.apache.org commented on HBASE-5547:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4633/#review6820
---



src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
https://reviews.apache.org/r/4633/#comment15202

done. Originally modeled after the catalog tracker in naming.



src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
https://reviews.apache.org/r/4633/#comment15203

done.



src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
https://reviews.apache.org/r/4633/#comment15204

done.



src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
https://reviews.apache.org/r/4633/#comment15205

done



src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
https://reviews.apache.org/r/4633/#comment15206

done.



src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
https://reviews.apache.org/r/4633/#comment15207

Would rather not, since in that case we would keep around a persistent 
connection to ZK, which is against what the community has been doing with 
cutting the client/zk connection (though admittedly that not directly 
applicable here).



src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
https://reviews.apache.org/r/4633/#comment15208

changed to confirmedArchiving.



src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
https://reviews.apache.org/r/4633/#comment15184

Yeah, it would. But then you should not attempt to expect the archive as 
valid and fail your backup process. Was going to leave it to the user to 
determine if they wanted to do the cleanup.

However, I would be ok if we added a method that cleaned up the filesystem 
for files created in the backup after the archiving started. 

Unfortunately, this means add an extra method on the master which I was 
trying to avoid, but I guess its not the end of the world. We also have a 
two-phase commit issue here as we also need to then write a 
'consistent-confirmed' file into the archive when it completes to ensure that 
failed clients don't break the archiving.

thoughts?



src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
https://reviews.apache.org/r/4633/#comment15209

done.



src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
https://reviews.apache.org/r/4633/#comment15183

Method name isn't enough?



src/main/java/org/apache/hadoop/hbase/client/HFileArchiveManager.java
https://reviews.apache.org/r/4633/#comment15210

done.



src/main/java/org/apache/hadoop/hbase/client/HFileArchiveManager.java
https://reviews.apache.org/r/4633/#comment15211

done.



src/main/java/org/apache/hadoop/hbase/client/HFileArchiveManager.java
https://reviews.apache.org/r/4633/#comment15212

done - regionsBeingArchived it is.



src/main/java/org/apache/hadoop/hbase/regionserver/HFileArchiveMonitor.java
https://reviews.apache.org/r/4633/#comment15213

done.



src/main/java/org/apache/hadoop/hbase/regionserver/HFileArchiveMonitor.java
https://reviews.apache.org/r/4633/#comment15185

Cruft from the earlier version.



src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
https://reviews.apache.org/r/4633/#comment15186

yeah, guess it should be.



src/main/java/org/apache/hadoop/hbase/util/HFileArchiveUtil.java
https://reviews.apache.org/r/4633/#comment15214

done.



src/main/java/org/apache/hadoop/hbase/util/HFileArchiveUtil.java
https://reviews.apache.org/r/4633/#comment15187

This is a super rare case where the name of the regiondir to be backed up 
is the same as the one in the backup already. Just being extra careful that the 
random storefile name doesn't match, though it is _very_ unlikely to happen, it 
still could (which could lead to bad circumstances). Currently, the storefile 
check doesn't check the archive directory and I think it would be pretty big 
overhead to expect it to do that as well (lots of added dependencies, calls, 
etc). We can just do the check and renaming here - its not super complicated 
and seems to work.



src/main/java/org/apache/hadoop/hbase/zookeeper/HFileArchiveTracker.java
https://reviews.apache.org/r/4633/#comment15215

done.



src/main/java/org/apache/hadoop/hbase/zookeeper/RegionServerHFileTracker.java
https://reviews.apache.org/r/4633/#comment15216

done.


- Jesse


On 2012-04-07 19:51:11, Jesse Yates wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4633/
bq.

[jira] [Commented] (HBASE-5547) Don't delete HFiles when in backup mode

2012-04-11 Thread jirapos...@reviews.apache.org (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251735#comment-13251735
]

Zhihong Yu commented on HBASE-5547:
---

bq. However, I would be ok if we added a method that cleaned up the filesystem
for files created in the backup after the archiving started.
I think we should do the cleanup if archiving cannot complete.
If a new method in master interface is needed, we can accommodate that.

Don't delete HFiles when in backup mode
-

Key: HBASE-5547
URL: https://issues.apache.org/jira/browse/HBASE-5547
Project: HBase
Issue Type: New Feature
Reporter: Lars Hofhansl
Assignee: Jesse Yates

This came up in a discussion I had with Stack.
It would be nice if HBase could be notified that a backup is in progress (via
a znode for example) and in that case either:
1. rename HFiles to be delete to file.bck
2. rename the HFiles into a special directory
3. rename them to a general trash directory (which would not need to be tied
to backup mode).
That way it should be able to get a consistent backup based on HFiles (HDFS
snapshots or hard links would be better options here, but we do not have
those).
#1 makes cleanup a bit harder.

[jira] [Commented] (HBASE-4336) Convert source tree into maven modules

2012-04-11 Thread Jesse Yates (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251737#comment-13251737
]

Jesse Yates commented on HBASE-4336:

bq. I'm excited about this one. I liked the notion by Mat Corgan that we'd have
an hbase-hregion module and then there'd be an hbase-wal so Li Pi can
experiment standalone w/ multiple WALs at the one time, etc., etc. The new
client would be one.

Yeah, its gonna be great!

bq. Or not. Just do hbase-common and hbase-security? Can roll hbase-security
back into common when time comes.

I'm leaning towards this - no assurances as to when security is going to be
rolled in and its really not a big deal to get rid of the module (and even not
that much pain for people with patches when it is as they just apply the
patches to a new root). Is that the right ticket number for rolling security
in? Doesn't seem directly applicable...

Convert source tree into maven modules
--

Key: HBASE-4336
URL: https://issues.apache.org/jira/browse/HBASE-4336
Project: HBase
Issue Type: Task
Components: build
Reporter: Gary Helmling
Priority: Critical
Fix For: 0.96.0

When we originally converted the build to maven we had a single core module
defined, but later reverted this to a module-less build for the sake of
simplicity.
It now looks like it's time to re-address this, as we have an actual need for
modules to:
* provide a trimmed down client library that applications can make use of
* more cleanly support building against different versions of Hadoop, in
place of some of the reflection machinations currently required
* incorporate the secure RPC engine that depends on some secure Hadoop classes
I propose we start simply by refactoring into two initial modules:
* core - common classes and utilities, and client-side code and interfaces
* server - master and region server implementations and supporting code
This would also lay the groundwork for incorporating the HBase security
features that have been developed. Once the module structure is in place,
security-related features could then be incorporated into a third module --
security -- after normal review and approval. The security module could
then depend on secure Hadoop, without modifying the dependencies of the rest
of the HBase code.

[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped mega regions


[ 
https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251743#comment-13251743
 ] 

jirapos...@reviews.apache.org commented on HBASE-5719:
--



bq.  On 2012-04-11 15:44:08, jmhsieh wrote:
bq.   
src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java, 
lines 384-392
bq.   
https://reviews.apache.org/r/4649/diff/2-3/?file=100674#file100674line384
bq.  
bq.   nit: would be easier to read if 
bq.   assertTrue((r1.equals(ac)  r2.equals(ae)) || (r1.equals(ae)  
r2.equals(ac)));

That's the original version actually.  However, the equals method somehow 
doesn't work.  I didn't try to implement the equals method in the SimpleRange 
class.


- Jimmy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4649/#review6851
---


On 2012-04-09 19:27:53, Jimmy Xiang wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4649/
bq.  ---
bq.  
bq.  (Updated 2012-04-09 19:27:53)
bq.  
bq.  
bq.  Review request for hbase and jmhsieh.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Make it configurable to sideline some regions in big overlapped groups 
which hbck doesn't handle currently.
bq.  
bq.  The regions chose to sideline are those which overlap with most other 
regions.
bq.  
bq.  
bq.  This addresses bug HBASE-5719.
bq.  https://issues.apache.org/jira/browse/HBASE-5719
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 54f9b21 
bq.src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java 
17678dd 
bq.
src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java 
ac3b225 
bq.  
bq.  Diff: https://reviews.apache.org/r/4649/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  mvn -PlocalTests -Dtest=TestHBaseFsck* clean test
bq.  
bq.  Also tested in real system to fix inconsistencies.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Jimmy
bq.  
bq.



 Enhance hbck to sideline overlapped mega regions
 

 Key: HBASE-5719
 URL: https://issues.apache.org/jira/browse/HBASE-5719
 Project: HBase
  Issue Type: New Feature
  Components: hbck
Affects Versions: 0.94.0, 0.96.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.96.0

 Attachments: hbase-5719.patch


 If there are too many regions in one overlapped group (by default, more than 
 10), hbck currently doesn't merge them since it takes time.
 In this case, we can sideline some regions in the group and break the 
 overlapping to fix the inconsistency.  Later on, sidelined regions can be 
 bulk loaded manually.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5767) Add the hbase shell table_att for any attribute


[ 
https://issues.apache.org/jira/browse/HBASE-5767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251740#comment-13251740
 ] 

Hadoop QA commented on HBASE-5767:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12522264/HBASE-5767-V2.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 1 new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.client.TestShell

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1478//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1478//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1478//console

This message is automatically generated.

 Add the hbase shell table_att for any attribute
 ---

 Key: HBASE-5767
 URL: https://issues.apache.org/jira/browse/HBASE-5767
 Project: HBase
  Issue Type: Improvement
  Components: shell
Reporter: Xing Shi
Priority: Minor
 Attachments: HBASE-5767-V2.patch, HBASE-5767.patch


 Now the HTableDescriptor supports setValue(String key, String value) method, 
 but the hbase shell not support it.
 May be like this:
 {quota}
 hbase(main):003:0 alter 'test', METHOD='table_att', 'key1'='value1'
 Updating all regions with the new schema...
 1/1 regions updated.
 Done.
 0 row(s) in 1.0820 seconds
 hbase(main):005:0 describe 'test'
 DESCRIPTION   
 ENABLED  
  {NAME = 'test', key1 = 'value1', FAMILIES = [{NAME = 'f1', BLOOMFILTER 
 = 'NONE', RE true 
  PLICATION_SCOPE = '0', VERSIONS = '3', COMPRESSION = 'NONE', MIN_VERSIONS 
 = '0', TTL  
   = '2147483647', BLOCKSIZE = '65536', IN_MEMORY = 'false', BLOCKCACHE = 
 'true'}]} 
 1 row(s) in 0.0300 seconds
 hbase(main):007:0 alter 'test', METHOD='table_att_unset', NAME='key1'
 Updating all regions with the new schema...
 1/1 regions updated.
 Done.
 0 row(s) in 1.0860 seconds
 hbase(main):008:0 describe 'test'
 DESCRIPTION   
 ENABLED  
  {NAME = 'test', FAMILIES = [{NAME = 'f1', BLOOMFILTER = 'NONE', 
 REPLICATION_SCOPE = false
   '0', VERSIONS = '3', COMPRESSION = 'NONE', MIN_VERSIONS = '0', TTL = 
 '2147483647',   
  BLOCKSIZE = '65536', IN_MEMORY = 'false', BLOCKCACHE = 'true'}]}  
  
 1 row(s) in 0.0280 seconds
 {quota}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5599) [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies

[
https://issues.apache.org/jira/browse/HBASE-5599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Hsieh updated HBASE-5599:
--

Resolution: Fixed
Fix Version/s: 0.96.0
0.94.0
0.92.2
0.90.7
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)

Fulin, thanks for the patch!

Committed to 0.90/0.92/0.94/0.96-trunk

[hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies

Attachments: 0.90-surefire-report-hbck.html, hbase-5599-0.90.patch,
hbase-5599-0.90_v2.patch, hbase-5599-0.90_v3.patch, hbase-5599-0.90_v5.patch,
hbase-5599-0.90_v6.patch, hbase-5599-0.90_v7.patch, hbase-5599-0.90_v8,
hbase-5599-0.92_v5.patch, hbase-5599-0.94_v5.patch, hbase-5599-92-v8.patch,
hbase-5599-trunk_v5.patch, hbase-5599-trunk_v7.patch,
hbase-5599-trunk_v8.patch, license.png

The hbck tool can not fix the six scenarios.
1. Version file does not exist in root dir.
Fix: I try to create a version file by 'FSUtils.setVersion' method.

2. [REGIONNAME][KEY] on HDFS, but not listed in META or deployed on any
region server.
Fix: I get region info form the hdfs file, this region info write to
'.META.' table.

3. [REGIONNAME][KEY] not in META, but deployed on [SERVERNAME]
Fix: I get region info form the hdfs file, this region info write to
'.META.' table.

4. [REGIONNAME] should not be deployed according to META, but is deployed on
[SERVERNAME]
Fix: Close this region.

[jira] [Updated] (HBASE-5599) [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies

2012-04-11 Thread jirapos...@reviews.apache.org (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Hsieh updated HBASE-5599:
--

Attachment: hbase-5599-92-v8.patch

Attached tweaked patch applied to 0.92

[hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies

The hbck tool can not fix the six scenarios.
1. Version file does not exist in root dir.
Fix: I try to create a version file by 'FSUtils.setVersion' method.

2. [REGIONNAME][KEY] on HDFS, but not listed in META or deployed on any
region server.
Fix: I get region info form the hdfs file, this region info write to
'.META.' table.

3. [REGIONNAME][KEY] not in META, but deployed on [SERVERNAME]
Fix: I get region info form the hdfs file, this region info write to
'.META.' table.

4. [REGIONNAME] should not be deployed according to META, but is deployed on
[SERVERNAME]
Fix: Close this region.

[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped mega regions


[ 
https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251768#comment-13251768
 ] 

jirapos...@reviews.apache.org commented on HBASE-5719:
--



bq.  On 2012-04-11 15:44:08, jmhsieh wrote:
bq.   
src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java, 
lines 384-392
bq.   
https://reviews.apache.org/r/4649/diff/2-3/?file=100674#file100674line384
bq.  
bq.   nit: would be easier to read if 
bq.   assertTrue((r1.equals(ac)  r2.equals(ae)) || (r1.equals(ae)  
r2.equals(ac)));
bq.  
bq.  Jimmy Xiang wrote:
bq.  That's the original version actually.  However, the equals method 
somehow doesn't work.  I didn't try to implement the equals method in the 
SimpleRange class.

I see.  Either is fine by me -- as is or you can make the change.:)


- jmhsieh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4649/#review6851
---


On 2012-04-09 19:27:53, Jimmy Xiang wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4649/
bq.  ---
bq.  
bq.  (Updated 2012-04-09 19:27:53)
bq.  
bq.  
bq.  Review request for hbase and jmhsieh.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Make it configurable to sideline some regions in big overlapped groups 
which hbck doesn't handle currently.
bq.  
bq.  The regions chose to sideline are those which overlap with most other 
regions.
bq.  
bq.  
bq.  This addresses bug HBASE-5719.
bq.  https://issues.apache.org/jira/browse/HBASE-5719
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 54f9b21 
bq.src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java 
17678dd 
bq.
src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java 
ac3b225 
bq.  
bq.  Diff: https://reviews.apache.org/r/4649/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  mvn -PlocalTests -Dtest=TestHBaseFsck* clean test
bq.  
bq.  Also tested in real system to fix inconsistencies.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Jimmy
bq.  
bq.



 Enhance hbck to sideline overlapped mega regions
 

 Key: HBASE-5719
 URL: https://issues.apache.org/jira/browse/HBASE-5719
 Project: HBase
  Issue Type: New Feature
  Components: hbck
Affects Versions: 0.94.0, 0.96.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.96.0

 Attachments: hbase-5719.patch


 If there are too many regions in one overlapped group (by default, more than 
 10), hbck currently doesn't merge them since it takes time.
 In this case, we can sideline some regions in the group and break the 
 overlapping to fix the inconsistency.  Later on, sidelined regions can be 
 bulk loaded manually.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5719) Enhance hbck to sideline overlapped mega regions

2012-04-11 Thread Jimmy Xiang (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-5719:
---

Status: Open  (was: Patch Available)

 Enhance hbck to sideline overlapped mega regions
 

 Key: HBASE-5719
 URL: https://issues.apache.org/jira/browse/HBASE-5719
 Project: HBase
  Issue Type: New Feature
  Components: hbck
Affects Versions: 0.94.0, 0.96.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.96.0

 Attachments: hbase-5719.patch, hbase-5719_0.90.patch, 
 hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3.patch


 If there are too many regions in one overlapped group (by default, more than 
 10), hbck currently doesn't merge them since it takes time.
 In this case, we can sideline some regions in the group and break the 
 overlapping to fix the inconsistency.  Later on, sidelined regions can be 
 bulk loaded manually.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5719) Enhance hbck to sideline overlapped mega regions

2012-04-11 Thread Jimmy Xiang (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-5719:
---

Attachment: hbase-5719_v3.patch
hbase-5719_0.94.patch
hbase-5719_0.92.patch
hbase-5719_0.90.patch

 Enhance hbck to sideline overlapped mega regions
 

 Key: HBASE-5719
 URL: https://issues.apache.org/jira/browse/HBASE-5719
 Project: HBase
  Issue Type: New Feature
  Components: hbck
Affects Versions: 0.94.0, 0.96.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.96.0

 Attachments: hbase-5719.patch, hbase-5719_0.90.patch, 
 hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3.patch


 If there are too many regions in one overlapped group (by default, more than 
 10), hbck currently doesn't merge them since it takes time.
 In this case, we can sideline some regions in the group and break the 
 overlapping to fix the inconsistency.  Later on, sidelined regions can be 
 bulk loaded manually.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5737) Minor Improvements related to balancer.

2012-04-11 Thread ramkrishna.s.vasudevan (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5737:
--

Attachment: HBASE-5737_2.patch

Updated patch.

 Minor Improvements related to balancer.
 ---

 Key: HBASE-5737
 URL: https://issues.apache.org/jira/browse/HBASE-5737
 Project: HBase
  Issue Type: Improvement
  Components: master
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Attachments: HBASE-5737.patch, HBASE-5737_1.patch, HBASE-5737_2.patch


 Currently in Am.getAssignmentByTable()  we use a result map which is currenly 
 a hashmap.  It could be better if we have a treeMap.  Even in 
 MetaReader.fullScan we have the treeMap only so that we have the naming order 
 maintained. I felt this change could be very useful in cases where we are 
 extending the DefaultLoadBalancer.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5737) Minor Improvements related to balancer.

2012-04-11 Thread ramkrishna.s.vasudevan (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5737:
--

Status: Open  (was: Patch Available)

 Minor Improvements related to balancer.
 ---

 Key: HBASE-5737
 URL: https://issues.apache.org/jira/browse/HBASE-5737
 Project: HBase
  Issue Type: Improvement
  Components: master
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Attachments: HBASE-5737.patch, HBASE-5737_1.patch, HBASE-5737_2.patch


 Currently in Am.getAssignmentByTable()  we use a result map which is currenly 
 a hashmap.  It could be better if we have a treeMap.  Even in 
 MetaReader.fullScan we have the treeMap only so that we have the naming order 
 maintained. I felt this change could be very useful in cases where we are 
 extending the DefaultLoadBalancer.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5719) Enhance hbck to sideline overlapped mega regions

2012-04-11 Thread Jimmy Xiang (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-5719:
---

Hadoop Flags: Reviewed
  Status: Patch Available  (was: Open)

 Enhance hbck to sideline overlapped mega regions
 

 Key: HBASE-5719
 URL: https://issues.apache.org/jira/browse/HBASE-5719
 Project: HBase
  Issue Type: New Feature
  Components: hbck
Affects Versions: 0.94.0, 0.96.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.96.0

 Attachments: hbase-5719.patch, hbase-5719_0.90.patch, 
 hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3.patch


 If there are too many regions in one overlapped group (by default, more than 
 10), hbck currently doesn't merge them since it takes time.
 In this case, we can sideline some regions in the group and break the 
 overlapping to fix the inconsistency.  Later on, sidelined regions can be 
 bulk loaded manually.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5737) Minor Improvements related to balancer.

2012-04-11 Thread ramkrishna.s.vasudevan (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5737:
--

Status: Patch Available  (was: Open)

 Minor Improvements related to balancer.
 ---

 Key: HBASE-5737
 URL: https://issues.apache.org/jira/browse/HBASE-5737
 Project: HBase
  Issue Type: Improvement
  Components: master
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Attachments: HBASE-5737.patch, HBASE-5737_1.patch, HBASE-5737_2.patch


 Currently in Am.getAssignmentByTable()  we use a result map which is currenly 
 a hashmap.  It could be better if we have a treeMap.  Even in 
 MetaReader.fullScan we have the treeMap only so that we have the naming order 
 maintained. I felt this change could be very useful in cases where we are 
 extending the DefaultLoadBalancer.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5547) Don't delete HFiles when in backup mode


[ 
https://issues.apache.org/jira/browse/HBASE-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251769#comment-13251769
 ] 

Zhihong Yu commented on HBASE-5547:
---

For the recovery cleanup, can we remember the start time of archiving and only 
delete archive files which are written after this start time ?
I like the /hbase/.archive/[table] structure.

 Don't delete HFiles when in backup mode
 -

 Key: HBASE-5547
 URL: https://issues.apache.org/jira/browse/HBASE-5547
 Project: HBase
  Issue Type: New Feature
Reporter: Lars Hofhansl
Assignee: Jesse Yates

 This came up in a discussion I had with Stack.
 It would be nice if HBase could be notified that a backup is in progress (via 
 a znode for example) and in that case either:
 1. rename HFiles to be delete to file.bck
 2. rename the HFiles into a special directory
 3. rename them to a general trash directory (which would not need to be tied 
 to backup mode).
 That way it should be able to get a consistent backup based on HFiles (HDFS 
 snapshots or hard links would be better options here, but we do not have 
 those).
 #1 makes cleanup a bit harder.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped mega regions

[
https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251774#comment-13251774
]

Hadoop QA commented on HBASE-5719:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522283/hbase-5719_v3.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

-1 patch. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1479//console

This message is automatically generated.

Enhance hbck to sideline overlapped mega regions

Key: HBASE-5719
URL: https://issues.apache.org/jira/browse/HBASE-5719
Project: HBase
Issue Type: New Feature
Components: hbck
Affects Versions: 0.94.0, 0.96.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Fix For: 0.96.0

Attachments: hbase-5719.patch, hbase-5719_0.90.patch,
hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3.patch

If there are too many regions in one overlapped group (by default, more than
10), hbck currently doesn't merge them since it takes time.
In this case, we can sideline some regions in the group and break the
overlapping to fix the inconsistency. Later on, sidelined regions can be
bulk loaded manually.

[jira] [Commented] (HBASE-5547) Don't delete HFiles when in backup mode

2012-04-11 Thread Jesse Yates (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251775#comment-13251775
]

Jesse Yates commented on HBASE-5547:

bq. For the recovery cleanup, can we remember the start time of archiving and
only delete archive files which are written after this start time

totally. I was actually thinking this was the way to go about it last night and
completely forgot it this morning. Thanks for looking out :)

Don't delete HFiles when in backup mode
-

Key: HBASE-5547
URL: https://issues.apache.org/jira/browse/HBASE-5547
Project: HBase
Issue Type: New Feature
Reporter: Lars Hofhansl
Assignee: Jesse Yates

[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped mega regions

2012-04-11 Thread Uma Maheswara Rao G (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251791#comment-13251791
]

Hadoop QA commented on HBASE-5719:
--

-1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12522285/hbase-5719_v3-new.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

-1 patch. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1481//console

This message is automatically generated.

Enhance hbck to sideline overlapped mega regions

Attachments: hbase-5719.patch, hbase-5719_0.90.patch,
hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch,
hbase-5719_v3.patch

[jira] [Updated] (HBASE-5763) Fix random failures in TestFSErrorsExposed

2012-04-11 Thread Phabricator (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5763:
---

Attachment: D2739.3.patch

mbautin updated the revision [jira] [HBASE-5763] [89-fb] Fix random failures 
in TestFSErrorsExposed.
Reviewers: Liyin, Kannan, pritamdamania, JIRA

  Actually, the 10 second wait is not necessary. Everything works without it.

  Also, really removing the unused variable this time.

REVISION DETAIL
  https://reviews.facebook.net/D2739

AFFECTED FILES
  src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
  src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java


 Fix random failures in TestFSErrorsExposed
 --

 Key: HBASE-5763
 URL: https://issues.apache.org/jira/browse/HBASE-5763
 Project: HBase
  Issue Type: Bug
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin
Priority: Minor
 Attachments: D2739.1.patch, D2739.2.patch, D2739.3.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-5768) Question about configuring HBase file size

2012-04-11 Thread Jean-Daniel Cryans (Resolved) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jean-Daniel Cryans resolved HBASE-5768.
---

Resolution: Invalid

Please send your questions to the user mailing list: 
http://hbase.apache.org/mail-lists.html

This Jira is for HBase development only.

 Question about configuring HBase file size
 --

 Key: HBASE-5768
 URL: https://issues.apache.org/jira/browse/HBASE-5768
 Project: HBase
  Issue Type: Task
Reporter: Ionut Ignatescu

 My use-case: I have several HBase tables in which entries are pushed 
 constantly(300k entries/5min,6-7Gb/day). I started with tables splitted in 32 
 regions.
 File size is currently set to 1Gb. I'm using this tables to perfom real-time 
 scans via a web app. 
 Problem: Sometimes,but pretty frequently tables are frozen and I get scanner 
 timeout exception.I found in logs that many splits and compactions are 
 performing and I think this are the cause. If I stop pushing data into 
 tables, are scans are running ok.
 Could you provide me some advices to do in order to avoid this problem?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5645) [findbugs] Fix correctness warnings

2012-04-11 Thread Uma Maheswara Rao G (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uma Maheswara Rao G updated HBASE-5645:
---

Attachment: HBASE-5645-3.patch

 [findbugs] Fix correctness warnings
 ---

 Key: HBASE-5645
 URL: https://issues.apache.org/jira/browse/HBASE-5645
 Project: HBase
  Issue Type: Sub-task
  Components: scripts
Reporter: Jonathan Hsieh
Assignee: David S. Wang
 Attachments: HBASE-5645-2.patch, HBASE-5645-3.patch, 
 HBASE-5645.patch, HBASE-5645.patch, hbase-5645-2-tweak.patch


 See 
 https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
 Fix the warnings in the correctness section.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5645) [findbugs] Fix correctness warnings


[ 
https://issues.apache.org/jira/browse/HBASE-5645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251809#comment-13251809
 ] 

Uma Maheswara Rao G commented on HBASE-5645:


Attached the same patch as David and exclude Hlog file as that was already 
handled. also not included ShutDownHook and Store changes as they already 
handled. Also updated the count in test-patch.properties.

 [findbugs] Fix correctness warnings
 ---

 Key: HBASE-5645
 URL: https://issues.apache.org/jira/browse/HBASE-5645
 Project: HBase
  Issue Type: Sub-task
  Components: scripts
Reporter: Jonathan Hsieh
Assignee: David S. Wang
 Attachments: HBASE-5645-2.patch, HBASE-5645-3.patch, 
 HBASE-5645.patch, HBASE-5645.patch, hbase-5645-2-tweak.patch


 See 
 https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
 Fix the warnings in the correctness section.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5737) Minor Improvements related to balancer.

2012-04-11 Thread jirapos...@reviews.apache.org (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251815#comment-13251815
]

Hadoop QA commented on HBASE-5737:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522284/HBASE-5737_2.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestSplitLogManager

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1480//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1480//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1480//console

This message is automatically generated.

Minor Improvements related to balancer.
---

Key: HBASE-5737
URL: https://issues.apache.org/jira/browse/HBASE-5737
Project: HBase
Issue Type: Improvement
Components: master
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
Attachments: HBASE-5737.patch, HBASE-5737_1.patch, HBASE-5737_2.patch

Currently in Am.getAssignmentByTable() we use a result map which is currenly
a hashmap. It could be better if we have a treeMap. Even in
MetaReader.fullScan we have the treeMap only so that we have the naming order
maintained. I felt this change could be very useful in cases where we are
extending the DefaultLoadBalancer.

[jira] [Commented] (HBASE-5768) Question about configuring HBase file size

2012-04-11 Thread Ionut Ignatescu (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251825#comment-13251825
 ] 

Ionut Ignatescu commented on HBASE-5768:


Thanks!

 Question about configuring HBase file size
 --

 Key: HBASE-5768
 URL: https://issues.apache.org/jira/browse/HBASE-5768
 Project: HBase
  Issue Type: Task
Reporter: Ionut Ignatescu

 My use-case: I have several HBase tables in which entries are pushed 
 constantly(300k entries/5min,6-7Gb/day). I started with tables splitted in 32 
 regions.
 File size is currently set to 1Gb. I'm using this tables to perfom real-time 
 scans via a web app. 
 Problem: Sometimes,but pretty frequently tables are frozen and I get scanner 
 timeout exception.I found in logs that many splits and compactions are 
 performing and I think this are the cause. If I stop pushing data into 
 tables, are scans are running ok.
 Could you provide me some advices to do in order to avoid this problem?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4676) Prefix Compression - Trie data block encoding

2012-04-11 Thread Matt Corgan (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251836#comment-13251836
]

Matt Corgan commented on HBASE-4676:

the git url in above comment is wrong. reposting instructions with public
https endpoint (looks like i can't edit jira comments)

To use with Eclipse:

* check out hbase-0.94 from apache subversion to myworkspace
* apply the HBASE-4676-0.94-v1.patch from HBASE-4676
* cd myworkspace
* git clone -b 0.1 https://hotp...@github.com/hotpads/hbase-prefix-trie.git
hbase-prefix-trie-0.1
* in eclipse - new java project
* type hbase-prefix-trie-0.1 and eclipse should find it. continue
* on the Projects tab, add hbase-0.94. finish
* ctrl-shift-R to open SeekBenchmarkMain.java
* follow instructions in SeekBenchmarkMain

To build a jar after modifying code:

* edit package-hbase-prefix-trie.xml in the project root to point to your build
directory
* right click it - Run As - Ant Build

Prefix Compression - Trie data block encoding
-

Key: HBASE-4676
URL: https://issues.apache.org/jira/browse/HBASE-4676
Project: HBase
Issue Type: New Feature
Components: io, performance, regionserver
Affects Versions: 0.90.6
Reporter: Matt Corgan
Attachments: HBASE-4676-0.94-v1.patch, PrefixTrie_Format_v1.pdf,
PrefixTrie_Performance_v1.pdf, SeeksPerSec by blockSize.png,
hbase-prefix-trie-0.1.jar

The HBase data block format has room for 2 significant improvements for
applications that have high block cache hit ratios.
First, there is no prefix compression, and the current KeyValue format is
somewhat metadata heavy, so there can be tremendous memory bloat for many
common data layouts, specifically those with long keys and short values.
Second, there is no random access to KeyValues inside data blocks. This
means that every time you double the datablock size, average seek time (or
average cpu consumption) goes up by a factor of 2. The standard 64KB block
size is ~10x slower for random seeks than a 4KB block size, but block sizes
as small as 4KB cause problems elsewhere. Using block sizes of 256KB or 1MB
or more may be more efficient from a disk access and block-cache perspective
in many big-data applications, but doing so is infeasible from a random seek
perspective.
The PrefixTrie block encoding format attempts to solve both of these
problems. Some features:
* trie format for row key encoding completely eliminates duplicate row keys
and encodes similar row keys into a standard trie structure which also saves
a lot of space
* the column family is currently stored once at the beginning of each block.
this could easily be modified to allow multiple family names per block
* all qualifiers in the block are stored in their own trie format which
caters nicely to wide rows. duplicate qualifers between rows are eliminated.
the size of this trie determines the width of the block's qualifier
fixed-width-int
* the minimum timestamp is stored at the beginning of the block, and deltas
are calculated from that. the maximum delta determines the width of the
block's timestamp fixed-width-int
The block is structured with metadata at the beginning, then a section for
the row trie, then the column trie, then the timestamp deltas, and then then
all the values. Most work is done in the row trie, where every leaf node
(corresponding to a row) contains a list of offsets/references corresponding
to the cells in that row. Each cell is fixed-width to enable binary
searching and is represented by [1 byte operationType, X bytes qualifier
offset, X bytes timestamp delta offset].
If all operation types are the same for a block, there will be zero per-cell
overhead. Same for timestamps. Same for qualifiers when i get a chance.
So, the compression aspect is very strong, but makes a few small sacrifices
on VarInt size to enable faster binary searches in trie fan-out nodes.
A more compressed but slower version might build on this by also applying
further (suffix, etc) compression on the trie nodes at the cost of slower
write speed. Even further compression could be obtained by using all VInts
instead of FInts with a sacrifice on random seek speed (though not huge).
One current drawback is the current write speed. While programmed with good
constructs like TreeMaps, ByteBuffers, binary searches, etc, it's not
programmed with the same level of optimization as the read path. Work will
need to be done to optimize the data structures used for encoding and could
probably show a 10x increase. It will still be slower than delta encoding,
but with a much higher decode speed. I have not yet created a thorough
benchmark for write speed nor sequential read speed.
Though the

[jira] [Commented] (HBASE-5741) ImportTsv does not check for table existence


[ 
https://issues.apache.org/jira/browse/HBASE-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251851#comment-13251851
 ] 

jirapos...@reviews.apache.org commented on HBASE-5741:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4700/
---

(Updated 2012-04-11 19:01:39.660787)


Review request for hbase.


Changes
---

Changes done as per Ted.


Summary
---

There is a bulk output option in the importtsv workload. It outputs the HFiles 
in a user defined directory. The current code assumes that a table with its 
name equal to the given output directory exists, and throws an exception 
otherwise. Here is a patch for creating a table in case it doesn't exist.


This addresses bug HBase-5741.
https://issues.apache.org/jira/browse/HBase-5741


Diffs (updated)
-

  src/main/java/org/apache/hadoop/hbase/mapreduce/ImportTsv.java ab22fc4 
  src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java ac30a62 

Diff: https://reviews.apache.org/r/4700/diff


Testing
---

Added a new test for bulkoutput; All importtsv tests pass.


Thanks,

Himanshu



 ImportTsv does not check for table existence 
 -

 Key: HBASE-5741
 URL: https://issues.apache.org/jira/browse/HBASE-5741
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 0.90.4
Reporter: Clint Heath
Assignee: Himanshu Vashishtha
 Attachments: HBase-5741.patch


 The usage statement for the importtsv command to hbase claims this:
 Note: if you do not use this option, then the target table must already 
 exist in HBase (in reference to the importtsv.bulk.output command-line 
 option)
 The truth is, the table must exist no matter what, importtsv cannot and will 
 not create it for you.
 This is the case because the createSubmittableJob method of ImportTsv does 
 not even attempt to check if the table exists already, much less create it:
 (From org.apache.hadoop.hbase.mapreduce.ImportTsv.java)
 305 HTable table = new HTable(conf, tableName);
 The HTable method signature in use there assumes the table exists and runs a 
 meta scan on it:
 (From org.apache.hadoop.hbase.client.HTable.java)
 142 * Creates an object to access a HBase table.
 ...
 151 public HTable(Configuration conf, final String tableName)
 What we should do inside of createSubmittableJob is something similar to what 
 the completebulkloads command would do:
 (Taken from org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.java)
 690 boolean tableExists = this.doesTableExist(tableName);
 691 if (!tableExists) this.createTable(tableName,dirPath);
 Currently the docs are misleading, the table in fact must exist prior to 
 running importtsv. We should check if it exists rather than assume it's 
 already there and throw the below exception:
 12/03/14 17:15:42 WARN client.HConnectionManager$HConnectionImplementation: 
 Encountered problems when prefetch META table: 
 org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for 
 table: myTable2, row=myTable2,,99
   at 
 org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:150)
 ...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5645) [findbugs] Fix correctness warnings

[
https://issues.apache.org/jira/browse/HBASE-5645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251855#comment-13251855
]

Hadoop QA commented on HBASE-5645:
--

+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522291/HBASE-5645-3.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

+1 core tests. The patch passed unit tests in .

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1482//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1482//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1482//console

This message is automatically generated.

[findbugs] Fix correctness warnings
---

Key: HBASE-5645
URL: https://issues.apache.org/jira/browse/HBASE-5645
Project: HBase
Issue Type: Sub-task
Components: scripts
Reporter: Jonathan Hsieh
Assignee: David S. Wang
Attachments: HBASE-5645-2.patch, HBASE-5645-3.patch,
HBASE-5645.patch, HBASE-5645.patch, hbase-5645-2-tweak.patch

See
https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Fix the warnings in the correctness section.

[jira] [Commented] (HBASE-5599) [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies


[ 
https://issues.apache.org/jira/browse/HBASE-5599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251859#comment-13251859
 ] 

Hudson commented on HBASE-5599:
---

Integrated in HBase-TRUNK #2743 (See 
[https://builds.apache.org/job/HBase-TRUNK/2743/])
HBASE-5599 [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED 
inconsistencies (fulin wang) (Revision 1324881)

 Result = SUCCESS
jmhsieh : 
Files : 
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedWithDetailsException.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/hbck/HbckTestingUtil.java


 [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies
 

 Key: HBASE-5599
 URL: https://issues.apache.org/jira/browse/HBASE-5599
 Project: HBase
  Issue Type: New Feature
  Components: hbck
Affects Versions: 0.90.6
Reporter: fulin wang
Assignee: fulin wang
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: 0.90-surefire-report-hbck.html, hbase-5599-0.90.patch, 
 hbase-5599-0.90_v2.patch, hbase-5599-0.90_v3.patch, hbase-5599-0.90_v5.patch, 
 hbase-5599-0.90_v6.patch, hbase-5599-0.90_v7.patch, hbase-5599-0.90_v8, 
 hbase-5599-0.92_v5.patch, hbase-5599-0.94_v5.patch, hbase-5599-92-v8.patch, 
 hbase-5599-trunk_v5.patch, hbase-5599-trunk_v7.patch, 
 hbase-5599-trunk_v8.patch, license.png


 The hbck tool can not fix the six scenarios.
 1. Version file does not exist in root dir.
Fix: I try to create a version file by 'FSUtils.setVersion' method.

 2. [REGIONNAME][KEY] on HDFS, but not listed in META or deployed on any 
 region server.
Fix: I get region info form the hdfs file, this region info write to 
 '.META.' table.

 3. [REGIONNAME][KEY] not in META, but deployed on [SERVERNAME]
Fix: I get region info form the hdfs file, this region info write to 
 '.META.' table.

 4. [REGIONNAME] should not be deployed according to META, but is deployed on 
 [SERVERNAME]
Fix: Close this region.

 5. First region should start with an empty key.  You need to  create a new 
 region and regioninfo in HDFS to plug the hole.
Fix: The region info is not in hdfs and .META., so it create a empty 
 region for this error.
 6. There is a hole in the region chain between [KEY] and [KEY]. You need to 
 create a new regioninfo and region dir in hdfs to plug the hole.
   Fix: The region info is not in hdfs and .META., so it create a empty region 
 for this hole.
   

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5684) Make ProcessBasedLocalHBaseCluster run HDFS and make it more robust

2012-04-11 Thread Phabricator (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251860#comment-13251860
 ] 

Phabricator commented on HBASE-5684:


Liyin has commented on the revision [jira] [HBASE-5684] [89-fb] Make 
ProcessBasedLocalHBaseCluster run HDFS and make it more robust.

  Mikhail, nice work:) LGTM !

REVISION DETAIL
  https://reviews.facebook.net/D2709

BRANCH
  make_processbasedlocalhbasecluster_run_HBASE-5684_v10


 Make ProcessBasedLocalHBaseCluster run HDFS and make it more robust
 ---

 Key: HBASE-5684
 URL: https://issues.apache.org/jira/browse/HBASE-5684
 Project: HBase
  Issue Type: Improvement
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin
 Attachments: D2709.1.patch, D2709.2.patch, D2709.3.patch, 
 D2709.4.patch


 Currently ProcessBasedLocalHBaseCluster runs on top of raw local filesystem. 
 We need it to start a process-based HDFS cluster as well. We also need to 
 make the whole thing more stable so we can use it in unit tests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5741) ImportTsv does not check for table existence

2012-04-11 Thread Himanshu Vashishtha (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Himanshu Vashishtha updated HBASE-5741:
---

Attachment: HBase-5741-v2.patch

Patch version 2 after rb.

 ImportTsv does not check for table existence 
 -

 Key: HBASE-5741
 URL: https://issues.apache.org/jira/browse/HBASE-5741
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 0.90.4
Reporter: Clint Heath
Assignee: Himanshu Vashishtha
 Attachments: HBase-5741-v2.patch, HBase-5741.patch


 The usage statement for the importtsv command to hbase claims this:
 Note: if you do not use this option, then the target table must already 
 exist in HBase (in reference to the importtsv.bulk.output command-line 
 option)
 The truth is, the table must exist no matter what, importtsv cannot and will 
 not create it for you.
 This is the case because the createSubmittableJob method of ImportTsv does 
 not even attempt to check if the table exists already, much less create it:
 (From org.apache.hadoop.hbase.mapreduce.ImportTsv.java)
 305 HTable table = new HTable(conf, tableName);
 The HTable method signature in use there assumes the table exists and runs a 
 meta scan on it:
 (From org.apache.hadoop.hbase.client.HTable.java)
 142 * Creates an object to access a HBase table.
 ...
 151 public HTable(Configuration conf, final String tableName)
 What we should do inside of createSubmittableJob is something similar to what 
 the completebulkloads command would do:
 (Taken from org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.java)
 690 boolean tableExists = this.doesTableExist(tableName);
 691 if (!tableExists) this.createTable(tableName,dirPath);
 Currently the docs are misleading, the table in fact must exist prior to 
 running importtsv. We should check if it exists rather than assume it's 
 already there and throw the below exception:
 12/03/14 17:15:42 WARN client.HConnectionManager$HConnectionImplementation: 
 Encountered problems when prefetch META table: 
 org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for 
 table: myTable2, row=myTable2,,99
   at 
 org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:150)
 ...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5741) ImportTsv does not check for table existence


 [ 
https://issues.apache.org/jira/browse/HBASE-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5741:
--

Hadoop Flags: Reviewed
  Status: Patch Available  (was: Open)

 ImportTsv does not check for table existence 
 -

 Key: HBASE-5741
 URL: https://issues.apache.org/jira/browse/HBASE-5741
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 0.90.4
Reporter: Clint Heath
Assignee: Himanshu Vashishtha
 Attachments: HBase-5741-v2.patch, HBase-5741.patch


 The usage statement for the importtsv command to hbase claims this:
 Note: if you do not use this option, then the target table must already 
 exist in HBase (in reference to the importtsv.bulk.output command-line 
 option)
 The truth is, the table must exist no matter what, importtsv cannot and will 
 not create it for you.
 This is the case because the createSubmittableJob method of ImportTsv does 
 not even attempt to check if the table exists already, much less create it:
 (From org.apache.hadoop.hbase.mapreduce.ImportTsv.java)
 305 HTable table = new HTable(conf, tableName);
 The HTable method signature in use there assumes the table exists and runs a 
 meta scan on it:
 (From org.apache.hadoop.hbase.client.HTable.java)
 142 * Creates an object to access a HBase table.
 ...
 151 public HTable(Configuration conf, final String tableName)
 What we should do inside of createSubmittableJob is something similar to what 
 the completebulkloads command would do:
 (Taken from org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.java)
 690 boolean tableExists = this.doesTableExist(tableName);
 691 if (!tableExists) this.createTable(tableName,dirPath);
 Currently the docs are misleading, the table in fact must exist prior to 
 running importtsv. We should check if it exists rather than assume it's 
 already there and throw the below exception:
 12/03/14 17:15:42 WARN client.HConnectionManager$HConnectionImplementation: 
 Encountered problems when prefetch META table: 
 org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for 
 table: myTable2, row=myTable2,,99
   at 
 org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:150)
 ...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5684) Make ProcessBasedLocalHBaseCluster run HDFS and make it more robust

2012-04-11 Thread Phabricator (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251867#comment-13251867
]

Phabricator commented on HBASE-5684:

amirshim has commented on the revision [jira] [HBASE-5684] [89-fb] Make
ProcessBasedLocalHBaseCluster run HDFS and make it more robust.

Looks good.

As we talked, maybe we should use
[[http://docs.oracle.com/javase/7/docs/api/java/nio/file/WatchService.html |
WatchService]] for log tailer once we decide to move to Java 7.

And we might want to split HBaseTestingUtility.java at some point since it's
over 1500 lines.

REVISION DETAIL
https://reviews.facebook.net/D2709

BRANCH
make_processbasedlocalhbasecluster_run_HBASE-5684_v10

Make ProcessBasedLocalHBaseCluster run HDFS and make it more robust
---

Key: HBASE-5684
URL: https://issues.apache.org/jira/browse/HBASE-5684
Project: HBase
Issue Type: Improvement
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin
Attachments: D2709.1.patch, D2709.2.patch, D2709.3.patch,
D2709.4.patch

Currently ProcessBasedLocalHBaseCluster runs on top of raw local filesystem.
We need it to start a process-based HDFS cluster as well. We also need to
make the whole thing more stable so we can use it in unit tests.

[jira] [Updated] (HBASE-5763) Fix random failures in TestFSErrorsExposed

2012-04-11 Thread Phabricator (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5763:
---

Attachment: D2739.4.patch

mbautin updated the revision [jira] [HBASE-5763] [89-fb] Fix random failures 
in TestFSErrorsExposed.
Reviewers: Liyin, Kannan, pritamdamania, JIRA

  Further simplifying the fix. Moving the method to kill all regionservers and 
masters to MiniHBaseCluster.

REVISION DETAIL
  https://reviews.facebook.net/D2739

AFFECTED FILES
  src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
  src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java


 Fix random failures in TestFSErrorsExposed
 --

 Key: HBASE-5763
 URL: https://issues.apache.org/jira/browse/HBASE-5763
 Project: HBase
  Issue Type: Bug
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin
Priority: Minor
 Attachments: D2739.1.patch, D2739.2.patch, D2739.3.patch, 
 D2739.4.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5599) [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies


[ 
https://issues.apache.org/jira/browse/HBASE-5599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251874#comment-13251874
 ] 

Hudson commented on HBASE-5599:
---

Integrated in HBase-0.94 #102 (See 
[https://builds.apache.org/job/HBase-0.94/102/])
HBASE-5599 [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED 
inconsistencies (fulin wang) (Revision 1324880)

 Result = SUCCESS
jmhsieh : 
Files : 
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/HbckTestingUtil.java


 [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies
 

 Key: HBASE-5599
 URL: https://issues.apache.org/jira/browse/HBASE-5599
 Project: HBase
  Issue Type: New Feature
  Components: hbck
Affects Versions: 0.90.6
Reporter: fulin wang
Assignee: fulin wang
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: 0.90-surefire-report-hbck.html, hbase-5599-0.90.patch, 
 hbase-5599-0.90_v2.patch, hbase-5599-0.90_v3.patch, hbase-5599-0.90_v5.patch, 
 hbase-5599-0.90_v6.patch, hbase-5599-0.90_v7.patch, hbase-5599-0.90_v8, 
 hbase-5599-0.92_v5.patch, hbase-5599-0.94_v5.patch, hbase-5599-92-v8.patch, 
 hbase-5599-trunk_v5.patch, hbase-5599-trunk_v7.patch, 
 hbase-5599-trunk_v8.patch, license.png


 The hbck tool can not fix the six scenarios.
 1. Version file does not exist in root dir.
Fix: I try to create a version file by 'FSUtils.setVersion' method.

 2. [REGIONNAME][KEY] on HDFS, but not listed in META or deployed on any 
 region server.
Fix: I get region info form the hdfs file, this region info write to 
 '.META.' table.

 3. [REGIONNAME][KEY] not in META, but deployed on [SERVERNAME]
Fix: I get region info form the hdfs file, this region info write to 
 '.META.' table.

 4. [REGIONNAME] should not be deployed according to META, but is deployed on 
 [SERVERNAME]
Fix: Close this region.

 5. First region should start with an empty key.  You need to  create a new 
 region and regioninfo in HDFS to plug the hole.
Fix: The region info is not in hdfs and .META., so it create a empty 
 region for this error.
 6. There is a hole in the region chain between [KEY] and [KEY]. You need to 
 create a new regioninfo and region dir in hdfs to plug the hole.
   Fix: The region info is not in hdfs and .META., so it create a empty region 
 for this hole.
   

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5645) [findbugs] Fix correctness warnings


 [ 
https://issues.apache.org/jira/browse/HBASE-5645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5645:
--

   Resolution: Fixed
Fix Version/s: 0.96.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Lgtm.  Thanks David and thanks Uma and Stack for taking a look.  I've committed 
to trunk.

 [findbugs] Fix correctness warnings
 ---

 Key: HBASE-5645
 URL: https://issues.apache.org/jira/browse/HBASE-5645
 Project: HBase
  Issue Type: Sub-task
  Components: scripts
Reporter: Jonathan Hsieh
Assignee: David S. Wang
 Fix For: 0.96.0

 Attachments: HBASE-5645-2.patch, HBASE-5645-3.patch, 
 HBASE-5645.patch, HBASE-5645.patch, hbase-5645-2-tweak.patch


 See 
 https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
 Fix the warnings in the correctness section.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5763) Fix random failures in TestFSErrorsExposed

2012-04-11 Thread Phabricator (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251897#comment-13251897
 ] 

Phabricator commented on HBASE-5763:


stack has commented on the revision [jira] [HBASE-5763] [89-fb] Fix random 
failures in TestFSErrorsExposed.

  lgtm

REVISION DETAIL
  https://reviews.facebook.net/D2739


 Fix random failures in TestFSErrorsExposed
 --

 Key: HBASE-5763
 URL: https://issues.apache.org/jira/browse/HBASE-5763
 Project: HBase
  Issue Type: Bug
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin
Priority: Minor
 Attachments: D2739.1.patch, D2739.2.patch, D2739.3.patch, 
 D2739.4.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5677) The master never does balance because duplicate openhandled the one region


 [ 
https://issues.apache.org/jira/browse/HBASE-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5677:
--

Attachment: 5677-proposal.txt

 The master never does balance because duplicate openhandled the one region
 --

 Key: HBASE-5677
 URL: https://issues.apache.org/jira/browse/HBASE-5677
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: 5677-proposal.txt, HBASE-5677-90-v1.patch, 
 surefire-report_no_patched_v1.html, surefire-report_patched_v1.html


 If region be assigned When the master is doing initialization(before do 
 processFailover),the region will be duplicate openhandled.
 because the unassigned node in zookeeper will be handled again in 
 AssignmentManager#processFailover()
 it cause the region in RIT,thus the master never does balance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5677) The master never does balance because duplicate openhandled the one region


 [ 
https://issues.apache.org/jira/browse/HBASE-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5677:
--

Status: Patch Available  (was: Open)

Run Lars' proposal through Hadoop QA

 The master never does balance because duplicate openhandled the one region
 --

 Key: HBASE-5677
 URL: https://issues.apache.org/jira/browse/HBASE-5677
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: 5677-proposal.txt, HBASE-5677-90-v1.patch, 
 surefire-report_no_patched_v1.html, surefire-report_patched_v1.html


 If region be assigned When the master is doing initialization(before do 
 processFailover),the region will be duplicate openhandled.
 because the unassigned node in zookeeper will be handled again in 
 AssignmentManager#processFailover()
 it cause the region in RIT,thus the master never does balance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5756) we can change defalult File Appender to RFA instead of DRFA.

2012-04-11 Thread stack (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251950#comment-13251950
 ] 

stack commented on HBASE-5756:
--

Default is DRFA in 0.94 and before.  RFA after (0.96)

 we can change defalult File Appender to RFA instead of DRFA.
 

 Key: HBASE-5756
 URL: https://issues.apache.org/jira/browse/HBASE-5756
 Project: HBase
  Issue Type: Bug
Reporter: rohithsharma
Priority: Minor

 This can be a point of concern when on a certain day the logging happens more 
 because of more and more activity. In that case the log file for that day can 
 grow huge. These logs can not be opened for analysis since size is more.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5677) The master never does balance because duplicate openhandled the one region

[
https://issues.apache.org/jira/browse/HBASE-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251954#comment-13251954
]

Hadoop QA commented on HBASE-5677:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522314/5677-proposal.txt
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplication

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1484//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1484//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1484//console

This message is automatically generated.

The master never does balance because duplicate openhandled the one region
--

Key: HBASE-5677
URL: https://issues.apache.org/jira/browse/HBASE-5677
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Environment: 0.90
Reporter: xufeng
Assignee: xufeng
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

Attachments: 5677-proposal.txt, HBASE-5677-90-v1.patch,
surefire-report_no_patched_v1.html, surefire-report_patched_v1.html

If region be assigned When the master is doing initialization(before do
processFailover),the region will be duplicate openhandled.
because the unassigned node in zookeeper will be handled again in
AssignmentManager#processFailover()
it cause the region in RIT,thus the master never does balance.

[jira] [Commented] (HBASE-5599) [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies


[ 
https://issues.apache.org/jira/browse/HBASE-5599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13251956#comment-13251956
 ] 

Hudson commented on HBASE-5599:
---

Integrated in HBase-0.92 #367 (See 
[https://builds.apache.org/job/HBase-0.92/367/])
HBASE-5599 [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED 
inconsistencies (fulin wang) (Revision 1324879)

 Result = FAILURE
jmhsieh : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* 
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java
* 
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/hbck/HbckTestingUtil.java


 [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies
 

 Key: HBASE-5599
 URL: https://issues.apache.org/jira/browse/HBASE-5599
 Project: HBase
  Issue Type: New Feature
  Components: hbck
Affects Versions: 0.90.6
Reporter: fulin wang
Assignee: fulin wang
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: 0.90-surefire-report-hbck.html, hbase-5599-0.90.patch, 
 hbase-5599-0.90_v2.patch, hbase-5599-0.90_v3.patch, hbase-5599-0.90_v5.patch, 
 hbase-5599-0.90_v6.patch, hbase-5599-0.90_v7.patch, hbase-5599-0.90_v8, 
 hbase-5599-0.92_v5.patch, hbase-5599-0.94_v5.patch, hbase-5599-92-v8.patch, 
 hbase-5599-trunk_v5.patch, hbase-5599-trunk_v7.patch, 
 hbase-5599-trunk_v8.patch, license.png


 The hbck tool can not fix the six scenarios.
 1. Version file does not exist in root dir.
Fix: I try to create a version file by 'FSUtils.setVersion' method.

 2. [REGIONNAME][KEY] on HDFS, but not listed in META or deployed on any 
 region server.
Fix: I get region info form the hdfs file, this region info write to 
 '.META.' table.

 3. [REGIONNAME][KEY] not in META, but deployed on [SERVERNAME]
Fix: I get region info form the hdfs file, this region info write to 
 '.META.' table.

 4. [REGIONNAME] should not be deployed according to META, but is deployed on 
 [SERVERNAME]
Fix: Close this region.

 5. First region should start with an empty key.  You need to  create a new 
 region and regioninfo in HDFS to plug the hole.
Fix: The region info is not in hdfs and .META., so it create a empty 
 region for this error.
 6. There is a hole in the region chain between [KEY] and [KEY]. You need to 
 create a new regioninfo and region dir in hdfs to plug the hole.
   Fix: The region info is not in hdfs and .META., so it create a empty region 
 for this hole.
   

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5741) ImportTsv does not check for table existence


 [ 
https://issues.apache.org/jira/browse/HBASE-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5741:
--

Attachment: 5741-v3.txt

Patch v3 addresses latest review comments.

 ImportTsv does not check for table existence 
 -

 Key: HBASE-5741
 URL: https://issues.apache.org/jira/browse/HBASE-5741
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 0.90.4
Reporter: Clint Heath
Assignee: Himanshu Vashishtha
 Attachments: 5741-v3.txt, HBase-5741-v2.patch, HBase-5741.patch


 The usage statement for the importtsv command to hbase claims this:
 Note: if you do not use this option, then the target table must already 
 exist in HBase (in reference to the importtsv.bulk.output command-line 
 option)
 The truth is, the table must exist no matter what, importtsv cannot and will 
 not create it for you.
 This is the case because the createSubmittableJob method of ImportTsv does 
 not even attempt to check if the table exists already, much less create it:
 (From org.apache.hadoop.hbase.mapreduce.ImportTsv.java)
 305 HTable table = new HTable(conf, tableName);
 The HTable method signature in use there assumes the table exists and runs a 
 meta scan on it:
 (From org.apache.hadoop.hbase.client.HTable.java)
 142 * Creates an object to access a HBase table.
 ...
 151 public HTable(Configuration conf, final String tableName)
 What we should do inside of createSubmittableJob is something similar to what 
 the completebulkloads command would do:
 (Taken from org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.java)
 690 boolean tableExists = this.doesTableExist(tableName);
 691 if (!tableExists) this.createTable(tableName,dirPath);
 Currently the docs are misleading, the table in fact must exist prior to 
 running importtsv. We should check if it exists rather than assume it's 
 already there and throw the below exception:
 12/03/14 17:15:42 WARN client.HConnectionManager$HConnectionImplementation: 
 Encountered problems when prefetch META table: 
 org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for 
 table: myTable2, row=myTable2,,99
   at 
 org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:150)
 ...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5741) ImportTsv does not check for table existence

2012-04-11 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5741:
--

Fix Version/s: 0.96.0
   0.94.0

 ImportTsv does not check for table existence 
 -

 Key: HBASE-5741
 URL: https://issues.apache.org/jira/browse/HBASE-5741
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 0.90.4
Reporter: Clint Heath
Assignee: Himanshu Vashishtha
 Fix For: 0.94.0, 0.96.0

 Attachments: 5741-v3.txt, HBase-5741-v2.patch, HBase-5741.patch


 The usage statement for the importtsv command to hbase claims this:
 Note: if you do not use this option, then the target table must already 
 exist in HBase (in reference to the importtsv.bulk.output command-line 
 option)
 The truth is, the table must exist no matter what, importtsv cannot and will 
 not create it for you.
 This is the case because the createSubmittableJob method of ImportTsv does 
 not even attempt to check if the table exists already, much less create it:
 (From org.apache.hadoop.hbase.mapreduce.ImportTsv.java)
 305 HTable table = new HTable(conf, tableName);
 The HTable method signature in use there assumes the table exists and runs a 
 meta scan on it:
 (From org.apache.hadoop.hbase.client.HTable.java)
 142 * Creates an object to access a HBase table.
 ...
 151 public HTable(Configuration conf, final String tableName)
 What we should do inside of createSubmittableJob is something similar to what 
 the completebulkloads command would do:
 (Taken from org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.java)
 690 boolean tableExists = this.doesTableExist(tableName);
 691 if (!tableExists) this.createTable(tableName,dirPath);
 Currently the docs are misleading, the table in fact must exist prior to 
 running importtsv. We should check if it exists rather than assume it's 
 already there and throw the below exception:
 12/03/14 17:15:42 WARN client.HConnectionManager$HConnectionImplementation: 
 Encountered problems when prefetch META table: 
 org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for 
 table: myTable2, row=myTable2,,99
   at 
 org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:150)
 ...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5620) Convert the client protocol of HRegionInterface to PB


[ 
https://issues.apache.org/jira/browse/HBASE-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252005#comment-13252005
 ] 

jirapos...@reviews.apache.org commented on HBASE-5620:
--



bq.  On 2012-04-11 06:22:10, Michael Stack wrote:
bq.   I got as far as HTable.  Still have more to go.  This is some pretty 
great work Jimmy.  This stuff is hard.  One thought I was having that is a 
little related (but it can be safely ignored) is what if there were only one 
method on an HRegionServer and you gave it an array of Gets, Puts, Delete, 
Increment, pb messages and you just let the regionserver sort it out.  is that 
even posssible?  I'll do rest of review tomorrow.
bq.  
bq.  Jimmy Xiang wrote:
bq.  Thanks for reviewing. As to one method to handle all actions, it is 
supported by the multi call.  It is too complex to match the response.

Sorry, I don't get the last part?  The response is too hard to make when the 
query is lots of types?


bq.  On 2012-04-11 06:22:10, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java, line 691
bq.   https://reviews.apache.org/r/4629/diff/2/?file=100870#file100870line691
bq.  
bq.   I said it before but I like this code removal
bq.  
bq.  Jimmy Xiang wrote:
bq.  It is not used so I removed it.

Great


bq.  On 2012-04-11 06:22:10, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/client/HConnection.java, line 235
bq.   https://reviews.apache.org/r/4629/diff/2/?file=100873#file100873line235
bq.  
bq.   Does this feel right Jimmy?  Seems odd asking an HConnection for a 
ClientProtocol?  It feels like a ClientProtocol should have an HConnection?  Or 
that we'd put together a ClientProtocol + an HConnection for it to use in a 
different object altogether.  What you think?
bq.   
bq.   Maybe this is a question for another client implementation that we 
do later?   But would be interested if you had any ideas on this.
bq.  
bq.  Jimmy Xiang wrote:
bq.  I agree a ClientProtocol should have an HConnection.  We can hide the 
HConnection. Given a region location, we can create a ClientProtocol which 
manages its own HConnection.

Sounds better


- Michael


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4629/#review6833
---


On 2012-04-07 21:30:32, Jimmy Xiang wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4629/
bq.  ---
bq.  
bq.  (Updated 2012-04-07 21:30:32)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This is the client protocol part of region interface. The admin protocol 
part will be done in a different jira.
bq.  
bq.  The HRegionInterface is still there since the admin part is not done yet.  
The other reason is that in case some people still wants the old interface
bq.  
bq.  Filters, comparators and coprocessor parameters are still Writable.  They 
will be addressed in different jiras.
bq.  
bq.  The existing client interface is not changed so that we don't break 
existing clients.
bq.  
bq.  
bq.  This addresses bug HBASE-5620.
bq.  https://issues.apache.org/jira/browse/HBASE-5620
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 
bq.src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa 
bq.src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 
bq.src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 
bq.src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 
0b783ce 
bq.src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f 
bq.src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 
9903df3 
bq.src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad 
bq.src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab 
bq.src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java 
cbfa489 
bq.src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 
bq.src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 
bq.src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 
bq.src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 
bq.
src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 
19ae18c 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java 
PRE-CREATION 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java 
PRE-CREATION 
bq.

[jira] [Commented] (HBASE-5677) The master never does balance because duplicate openhandled the one region


[ 
https://issues.apache.org/jira/browse/HBASE-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252007#comment-13252007
 ] 

Zhihong Yu commented on HBASE-5677:
---

I saw the following in org.apache.hadoop.hbase.mapreduce.TestImportTsv.txt when 
I tested HBASE-5741 in 0.94, with the proposed patch in place:
{code}
Caused by: java.lang.RuntimeException: Master not initialized after 200 seconds
  at 
org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:206)
  at 
org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:422)
  at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:196)
{code}

 The master never does balance because duplicate openhandled the one region
 --

 Key: HBASE-5677
 URL: https://issues.apache.org/jira/browse/HBASE-5677
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: 5677-proposal.txt, HBASE-5677-90-v1.patch, 
 surefire-report_no_patched_v1.html, surefire-report_patched_v1.html


 If region be assigned When the master is doing initialization(before do 
 processFailover),the region will be duplicate openhandled.
 because the unassigned node in zookeeper will be handled again in 
 AssignmentManager#processFailover()
 it cause the region in RIT,thus the master never does balance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5741) ImportTsv does not check for table existence

[
https://issues.apache.org/jira/browse/HBASE-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252025#comment-13252025
]

Hadoop QA commented on HBASE-5741:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522338/5741-94.txt
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

-1 patch. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1486//console

This message is automatically generated.

ImportTsv does not check for table existence
-

Key: HBASE-5741
URL: https://issues.apache.org/jira/browse/HBASE-5741
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 0.90.4
Reporter: Clint Heath
Assignee: Himanshu Vashishtha
Fix For: 0.94.0, 0.96.0

Attachments: 5741-94.txt, 5741-v3.txt, HBase-5741-v2.patch,
HBase-5741.patch

The usage statement for the importtsv command to hbase claims this:
Note: if you do not use this option, then the target table must already
exist in HBase (in reference to the importtsv.bulk.output command-line
option)
The truth is, the table must exist no matter what, importtsv cannot and will
not create it for you.
This is the case because the createSubmittableJob method of ImportTsv does
not even attempt to check if the table exists already, much less create it:
(From org.apache.hadoop.hbase.mapreduce.ImportTsv.java)
305 HTable table = new HTable(conf, tableName);
The HTable method signature in use there assumes the table exists and runs a
meta scan on it:
(From org.apache.hadoop.hbase.client.HTable.java)
142 * Creates an object to access a HBase table.
...
151 public HTable(Configuration conf, final String tableName)
What we should do inside of createSubmittableJob is something similar to what
the completebulkloads command would do:
(Taken from org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.java)
690 boolean tableExists = this.doesTableExist(tableName);
691 if (!tableExists) this.createTable(tableName,dirPath);
Currently the docs are misleading, the table in fact must exist prior to
running importtsv. We should check if it exists rather than assume it's
already there and throw the below exception:
12/03/14 17:15:42 WARN client.HConnectionManager$HConnectionImplementation:
Encountered problems when prefetch META table:
org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for
table: myTable2, row=myTable2,,99
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:150)
...

[jira] [Updated] (HBASE-5741) ImportTsv does not check for table existence

2012-04-11 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5741:
--

Comment: was deleted

(was: -1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12522338/5741-94.txt
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

-1 patch.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1486//console

This message is automatically generated.)

 ImportTsv does not check for table existence 
 -

 Key: HBASE-5741
 URL: https://issues.apache.org/jira/browse/HBASE-5741
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 0.90.4
Reporter: Clint Heath
Assignee: Himanshu Vashishtha
 Fix For: 0.94.0, 0.96.0

 Attachments: 5741-94.txt, 5741-v3.txt, HBase-5741-v2.patch, 
 HBase-5741.patch


 The usage statement for the importtsv command to hbase claims this:
 Note: if you do not use this option, then the target table must already 
 exist in HBase (in reference to the importtsv.bulk.output command-line 
 option)
 The truth is, the table must exist no matter what, importtsv cannot and will 
 not create it for you.
 This is the case because the createSubmittableJob method of ImportTsv does 
 not even attempt to check if the table exists already, much less create it:
 (From org.apache.hadoop.hbase.mapreduce.ImportTsv.java)
 305 HTable table = new HTable(conf, tableName);
 The HTable method signature in use there assumes the table exists and runs a 
 meta scan on it:
 (From org.apache.hadoop.hbase.client.HTable.java)
 142 * Creates an object to access a HBase table.
 ...
 151 public HTable(Configuration conf, final String tableName)
 What we should do inside of createSubmittableJob is something similar to what 
 the completebulkloads command would do:
 (Taken from org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.java)
 690 boolean tableExists = this.doesTableExist(tableName);
 691 if (!tableExists) this.createTable(tableName,dirPath);
 Currently the docs are misleading, the table in fact must exist prior to 
 running importtsv. We should check if it exists rather than assume it's 
 already there and throw the below exception:
 12/03/14 17:15:42 WARN client.HConnectionManager$HConnectionImplementation: 
 Encountered problems when prefetch META table: 
 org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for 
 table: myTable2, row=myTable2,,99
   at 
 org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:150)
 ...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5747) Forward port hbase-5708 [89-fb] Make MiniMapRedCluster directory a subdirectory of target/test

2012-04-11 Thread stack (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5747:
-

Status: Open  (was: Patch Available)

 Forward port hbase-5708 [89-fb] Make MiniMapRedCluster directory a 
 subdirectory of target/test
 

 Key: HBASE-5747
 URL: https://issues.apache.org/jira/browse/HBASE-5747
 Project: HBase
  Issue Type: Task
Reporter: stack
Assignee: stack
Priority: Blocker
 Attachments: 5474.txt, 5474v2.txt, 5474v3.txt


 Forward port as much as we can of Mikhail's hard-won test cleanups over on 
 0.89 branch  Will improve our being able to run unit tests in //.  He also 
 found a few bugs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5747) Forward port hbase-5708 [89-fb] Make MiniMapRedCluster directory a subdirectory of target/test

2012-04-11 Thread stack (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5747:
-

Attachment: 5474v3.txt

A few of the faillures were legit.  This patch includes fixes.

 Forward port hbase-5708 [89-fb] Make MiniMapRedCluster directory a 
 subdirectory of target/test
 

 Key: HBASE-5747
 URL: https://issues.apache.org/jira/browse/HBASE-5747
 Project: HBase
  Issue Type: Task
Reporter: stack
Assignee: stack
Priority: Blocker
 Attachments: 5474.txt, 5474v2.txt, 5474v3.txt


 Forward port as much as we can of Mikhail's hard-won test cleanups over on 
 0.89 branch  Will improve our being able to run unit tests in //.  He also 
 found a few bugs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5620) Convert the client protocol of HRegionInterface to PB


[ 
https://issues.apache.org/jira/browse/HBASE-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252035#comment-13252035
 ] 

jirapos...@reviews.apache.org commented on HBASE-5620:
--



bq.  On 2012-04-11 06:22:10, Michael Stack wrote:
bq.   I got as far as HTable.  Still have more to go.  This is some pretty 
great work Jimmy.  This stuff is hard.  One thought I was having that is a 
little related (but it can be safely ignored) is what if there were only one 
method on an HRegionServer and you gave it an array of Gets, Puts, Delete, 
Increment, pb messages and you just let the regionserver sort it out.  is that 
even posssible?  I'll do rest of review tomorrow.
bq.  
bq.  Jimmy Xiang wrote:
bq.  Thanks for reviewing. As to one method to handle all actions, it is 
supported by the multi call.  It is too complex to match the response.
bq.  
bq.  Michael Stack wrote:
bq.  Sorry, I don't get the last part?  The response is too hard to make 
when the query is lots of types?

Right, basically the code is not efficient.


- Jimmy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4629/#review6833
---


On 2012-04-07 21:30:32, Jimmy Xiang wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4629/
bq.  ---
bq.  
bq.  (Updated 2012-04-07 21:30:32)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This is the client protocol part of region interface. The admin protocol 
part will be done in a different jira.
bq.  
bq.  The HRegionInterface is still there since the admin part is not done yet.  
The other reason is that in case some people still wants the old interface
bq.  
bq.  Filters, comparators and coprocessor parameters are still Writable.  They 
will be addressed in different jiras.
bq.  
bq.  The existing client interface is not changed so that we don't break 
existing clients.
bq.  
bq.  
bq.  This addresses bug HBASE-5620.
bq.  https://issues.apache.org/jira/browse/HBASE-5620
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 
bq.src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa 
bq.src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 
bq.src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 
bq.src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 
0b783ce 
bq.src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f 
bq.src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 
9903df3 
bq.src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad 
bq.src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab 
bq.src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java 
cbfa489 
bq.src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 
bq.src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 
bq.src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 
bq.src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 
bq.
src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 
19ae18c 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java 
PRE-CREATION 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java 
PRE-CREATION 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 
PRE-CREATION 
bq.src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java 
PRE-CREATION 
bq.
src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 
PRE-CREATION 
bq.
src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 
PRE-CREATION 
bq.
src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 
4026da0 
bq.
src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 
2169310 
bq.
src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java
 b36a9c0 
bq.src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 
8a61f7d 
bq.
src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 
703e73d 
bq.src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f 
bq.src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 
PRE-CREATION 
bq.src/main/protobuf/Admin.proto PRE-CREATION 
bq.

[jira] [Commented] (HBASE-5741) ImportTsv does not check for table existence

[
https://issues.apache.org/jira/browse/HBASE-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252036#comment-13252036
]

Hadoop QA commented on HBASE-5741:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522330/5741-v3.txt
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1485//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1485//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1485//console

This message is automatically generated.

ImportTsv does not check for table existence
-

Attachments: 5741-94.txt, 5741-v3.txt, HBase-5741-v2.patch,
HBase-5741.patch

[jira] [Commented] (HBASE-5741) ImportTsv does not check for table existence


[ 
https://issues.apache.org/jira/browse/HBASE-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13252038#comment-13252038
 ] 

Zhihong Yu commented on HBASE-5741:
---

I would integrate patches to trunk and 0.94 tomorrow morning if there is no 
objection.

 ImportTsv does not check for table existence 
 -

 Key: HBASE-5741
 URL: https://issues.apache.org/jira/browse/HBASE-5741
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 0.90.4
Reporter: Clint Heath
Assignee: Himanshu Vashishtha
 Fix For: 0.94.0, 0.96.0

 Attachments: 5741-94.txt, 5741-v3.txt, HBase-5741-v2.patch, 
 HBase-5741.patch


 The usage statement for the importtsv command to hbase claims this:
 Note: if you do not use this option, then the target table must already 
 exist in HBase (in reference to the importtsv.bulk.output command-line 
 option)
 The truth is, the table must exist no matter what, importtsv cannot and will 
 not create it for you.
 This is the case because the createSubmittableJob method of ImportTsv does 
 not even attempt to check if the table exists already, much less create it:
 (From org.apache.hadoop.hbase.mapreduce.ImportTsv.java)
 305 HTable table = new HTable(conf, tableName);
 The HTable method signature in use there assumes the table exists and runs a 
 meta scan on it:
 (From org.apache.hadoop.hbase.client.HTable.java)
 142 * Creates an object to access a HBase table.
 ...
 151 public HTable(Configuration conf, final String tableName)
 What we should do inside of createSubmittableJob is something similar to what 
 the completebulkloads command would do:
 (Taken from org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.java)
 690 boolean tableExists = this.doesTableExist(tableName);
 691 if (!tableExists) this.createTable(tableName,dirPath);
 Currently the docs are misleading, the table in fact must exist prior to 
 running importtsv. We should check if it exists rather than assume it's 
 already there and throw the below exception:
 12/03/14 17:15:42 WARN client.HConnectionManager$HConnectionImplementation: 
 Encountered problems when prefetch META table: 
 org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for 
 table: myTable2, row=myTable2,,99
   at 
 org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:150)
 ...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5741) ImportTsv does not check for table existence


 [ 
https://issues.apache.org/jira/browse/HBASE-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5741:
--

Attachment: 5741-v3.txt

 ImportTsv does not check for table existence 
 -

 Key: HBASE-5741
 URL: https://issues.apache.org/jira/browse/HBASE-5741
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 0.90.4
Reporter: Clint Heath
Assignee: Himanshu Vashishtha
 Fix For: 0.94.0, 0.96.0

 Attachments: 5741-94.txt, 5741-v3.txt, HBase-5741-v2.patch, 
 HBase-5741.patch


 The usage statement for the importtsv command to hbase claims this:
 Note: if you do not use this option, then the target table must already 
 exist in HBase (in reference to the importtsv.bulk.output command-line 
 option)
 The truth is, the table must exist no matter what, importtsv cannot and will 
 not create it for you.
 This is the case because the createSubmittableJob method of ImportTsv does 
 not even attempt to check if the table exists already, much less create it:
 (From org.apache.hadoop.hbase.mapreduce.ImportTsv.java)
 305 HTable table = new HTable(conf, tableName);
 The HTable method signature in use there assumes the table exists and runs a 
 meta scan on it:
 (From org.apache.hadoop.hbase.client.HTable.java)
 142 * Creates an object to access a HBase table.
 ...
 151 public HTable(Configuration conf, final String tableName)
 What we should do inside of createSubmittableJob is something similar to what 
 the completebulkloads command would do:
 (Taken from org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.java)
 690 boolean tableExists = this.doesTableExist(tableName);
 691 if (!tableExists) this.createTable(tableName,dirPath);
 Currently the docs are misleading, the table in fact must exist prior to 
 running importtsv. We should check if it exists rather than assume it's 
 already there and throw the below exception:
 12/03/14 17:15:42 WARN client.HConnectionManager$HConnectionImplementation: 
 Encountered problems when prefetch META table: 
 org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for 
 table: myTable2, row=myTable2,,99
   at 
 org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:150)
 ...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5741) ImportTsv does not check for table existence


 [ 
https://issues.apache.org/jira/browse/HBASE-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5741:
--

Attachment: (was: 5741-v3.txt)

 ImportTsv does not check for table existence 
 -

 Key: HBASE-5741
 URL: https://issues.apache.org/jira/browse/HBASE-5741
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 0.90.4
Reporter: Clint Heath
Assignee: Himanshu Vashishtha
 Fix For: 0.94.0, 0.96.0

 Attachments: 5741-94.txt, 5741-v3.txt, HBase-5741-v2.patch, 
 HBase-5741.patch


 The usage statement for the importtsv command to hbase claims this:
 Note: if you do not use this option, then the target table must already 
 exist in HBase (in reference to the importtsv.bulk.output command-line 
 option)
 The truth is, the table must exist no matter what, importtsv cannot and will 
 not create it for you.
 This is the case because the createSubmittableJob method of ImportTsv does 
 not even attempt to check if the table exists already, much less create it:
 (From org.apache.hadoop.hbase.mapreduce.ImportTsv.java)
 305 HTable table = new HTable(conf, tableName);
 The HTable method signature in use there assumes the table exists and runs a 
 meta scan on it:
 (From org.apache.hadoop.hbase.client.HTable.java)
 142 * Creates an object to access a HBase table.
 ...
 151 public HTable(Configuration conf, final String tableName)
 What we should do inside of createSubmittableJob is something similar to what 
 the completebulkloads command would do:
 (Taken from org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.java)
 690 boolean tableExists = this.doesTableExist(tableName);
 691 if (!tableExists) this.createTable(tableName,dirPath);
 Currently the docs are misleading, the table in fact must exist prior to 
 running importtsv. We should check if it exists rather than assume it's 
 already there and throw the below exception:
 12/03/14 17:15:42 WARN client.HConnectionManager$HConnectionImplementation: 
 Encountered problems when prefetch META table: 
 org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for 
 table: myTable2, row=myTable2,,99
   at 
 org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:150)
 ...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5736) ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly


 [ 
https://issues.apache.org/jira/browse/HBASE-5736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5736:
--

Attachment: 5736-94.txt

Patch for 0.94

 ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly
 -

 Key: HBASE-5736
 URL: https://issues.apache.org/jira/browse/HBASE-5736
 Project: HBase
  Issue Type: Bug
Reporter: Scott Chen
Assignee: Scott Chen
 Fix For: 0.96.0

 Attachments: 5736-94.txt, HBASE-5736.D2649.1.patch, 
 HBASE-5736.D2649.2.patch, HBASE-5736.D2649.3.patch


 We have fixed similar bug in
 https://issues.apache.org/jira/browse/HBASE-5507
 It uses ByteBuffer.array() to read the ByteBuffer.
 This will ignore the offset return the whole underlying byte array.
 The bug can be triggered by using framed Transport thrift servers.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5736) ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly


 [ 
https://issues.apache.org/jira/browse/HBASE-5736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5736:
--

Status: Open  (was: Patch Available)

 ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly
 -

 Key: HBASE-5736
 URL: https://issues.apache.org/jira/browse/HBASE-5736
 Project: HBase
  Issue Type: Bug
Reporter: Scott Chen
Assignee: Scott Chen
 Fix For: 0.96.0

 Attachments: 5736-94.txt, HBASE-5736.D2649.1.patch, 
 HBASE-5736.D2649.2.patch, HBASE-5736.D2649.3.patch


 We have fixed similar bug in
 https://issues.apache.org/jira/browse/HBASE-5507
 It uses ByteBuffer.array() to read the ByteBuffer.
 This will ignore the offset return the whole underlying byte array.
 The bug can be triggered by using framed Transport thrift servers.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5736) ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly