[jira] [Updated] (HBASE-5535) Make the functions in task monitor synchronized

2012-03-30 Thread stack (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5535:
-

Fix Version/s: 0.92.2

Committed to 0.92 branch at Jon's prompting (thanks for the reminder)

 Make the functions in task monitor synchronized
 ---

 Key: HBASE-5535
 URL: https://issues.apache.org/jira/browse/HBASE-5535
 Project: HBase
  Issue Type: Bug
Reporter: Liyin Tang
Assignee: Liyin Tang
 Fix For: 0.92.2, 0.94.0

 Attachments: 
 HBASE-5535-Make-the-functions-in-task-monitor-synchr-2012-03-08_16_33_42.patch


 There are some potential race condition in the task monitor. So update the 
 functions in task monitor to be synchronized.
 The example of the problem caused by the race condition:
 ERROR org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Cache flush 
 failed for region 
 java.lang.IndexOutOfBoundsException: Index: 1745, Size: 1744
 at java.util.ArrayList.add(ArrayList.java:367)
 at java.util.SubList.add(AbstractList.java:633)
 at java.util.SubList.add(AbstractList.java:633)
 at java.util.SubList.add(AbstractList.java:633)
 at java.util.SubList.add(AbstractList.java:633)
 at java.util.SubList.add(AbstractList.java:633)
 at java.util.AbstractList.add(AbstractList.java:91)
 at 
 org.apache.hadoop.hbase.monitoring.TaskMonitor.createStatus(TaskMonitor.java:74)
 at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1139)
 at 
 org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:260)
 at 
 org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:234)
 at 
 org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:146)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5674) add support in HBase to overwrite hbase timestamp to a version number during major compaction

2012-03-30 Thread He Yongqiang (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242094#comment-13242094
 ] 

He Yongqiang commented on HBASE-5674:
-

I use the term 'researchy' as it is mentioned so in one email thread. refer to 
http://osdir.com/ml/general/2012-03/msg52707.html I have no idea how this term 
come up.

bq. The most of us working on hbase are trying to make it an hardcore 
production worthy platform. 'Pluggable' and 'research', at least on first 
blush, sound like distractions from the project objective.
So are you referring this as conflicting with your 'hardcore production worthy 
platform' goal? 

 add support in HBase to overwrite hbase timestamp to a version number during 
 major compaction
 -

 Key: HBASE-5674
 URL: https://issues.apache.org/jira/browse/HBASE-5674
 Project: HBase
  Issue Type: Improvement
Reporter: He Yongqiang
Assignee: He Yongqiang

 Right now, a millisecond-level timestamp is attached to every record. 
 In our case, we only need a version number (mostly it will be just zero etc). 
 A millisecond timestamp is too heavy to carry. We should add support to 
 overwrite it to zero during major compaction. 
 KVs before major compaction will remain using system timestamp. And this 
 should be configurable, so that we should not mess up if the hbase timestamp 
 is specified by application.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5674) add support in HBase to overwrite hbase timestamp to a version number during major compaction

2012-03-30 Thread stack (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242102#comment-13242102
 ] 

stack commented on HBASE-5674:
--

I didn't get the 'reference'.  Sorry, that email went over my head.

bq. So are you referring this as conflicting with your 'hardcore production 
worthy platform' goal?

First its not 'my' goal.   Check out the notes from recent HBase PMC meeting:  
http://blogs.apache.org/hbase/.

That said, I thought it a different kinda 'researchy' that was being referred 
to.  I like the 'research' you fellas are at.

(Pardon me.  I did not read the name on the issue before responding.  All I saw 
was the short description asking for an 'odd', little-substantiated behavior 
and I asked a question.  Even on your comeback, I failed to check the whom and 
was innocently reacting to what I thought was a request that hbase become a 
dumping ground for plugins and research)

 

 add support in HBase to overwrite hbase timestamp to a version number during 
 major compaction
 -

 Key: HBASE-5674
 URL: https://issues.apache.org/jira/browse/HBASE-5674
 Project: HBase
  Issue Type: Improvement
Reporter: He Yongqiang
Assignee: He Yongqiang

 Right now, a millisecond-level timestamp is attached to every record. 
 In our case, we only need a version number (mostly it will be just zero etc). 
 A millisecond timestamp is too heavy to carry. We should add support to 
 overwrite it to zero during major compaction. 
 KVs before major compaction will remain using system timestamp. And this 
 should be configurable, so that we should not mess up if the hbase timestamp 
 is specified by application.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HBASE-5677) The master never does balance because duplicate openhandled the one region

2012-03-30 Thread xufeng (Created) (JIRA)
The master never does balance because duplicate openhandled the one region
--

 Key: HBASE-5677
 URL: https://issues.apache.org/jira/browse/HBASE-5677
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90
Reporter: xufeng
Assignee: xufeng


If region be assigned When the master is doing initialization(before do 
processFailover),the region will be duplicate openhandled.
it cause the region in RIT,thus the master never does balance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5673) The OOM problem of IPC client call cause all handle block

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242112#comment-13242112
 ] 

Hudson commented on HBASE-5673:
---

Integrated in HBase-0.94 #69 (See 
[https://builds.apache.org/job/HBase-0.94/69/])
HBASE-5673 The OOM problem of IPC client call cause all handle block; 
REAPPLY; V2 OF PATCH (Revision 1307276)
HBASE-5673 The OOM problem of IPC client call cause all handle block; REVERT -- 
APPLIED WRONG PATCH (Revision 1307271)

 Result = FAILURE
stack : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java

stack : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java


 The OOM problem of IPC client call  cause all handle block
 --

 Key: HBASE-5673
 URL: https://issues.apache.org/jira/browse/HBASE-5673
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90.6
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2, 0.94.1

 Attachments: HBASE-5673-90-V2.patch, HBASE-5673-90.patch


 if HBaseClient meet unable to create new native thread exception, the call 
 will never complete because it be lost in calls queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5599) The hbck tool can not fix the six scenarios, it is NO_VERSION_FILE, NOT_IN_META_OR_DEPLOYED, NOT_IN_META, SHOULD_NOT_BE_DEPLOYED, FIRST_REGION_STARTKEY_NOT_EMPTY, HOLE_IN

2012-03-30 Thread fulin wang (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

fulin wang updated HBASE-5599:
--

Attachment: hbase-5599-0.90_v6.patch

I make a hbase-5599-0.90_v6.patch, please review.
This is test result:
Tests Errors  Failures Skipped Success Rate Time 
14 0 0 0 100% 181.667 

TestHBaseFsck testHBaseFsckClean 4.843 
 testDupeStartKey 10.086 
 testDupeRegion 10.382 
 testDegenerateRegions 10.098 
 testContainedRegionOverlap 9.598 
 testOverlapAndOrphan 12.687 
 testCoveredStartKey 10.836 
 testRegionHole 8.652 
 testHDFSRegioninfoMissing 11.895 
 testNotInMetaOrDeployedHole 8.262 
 testNotInMetaHole 8.387 
 testNotInHdfs 8.345 
 testNoHdfsTable 46.275

 The hbck tool can not fix the six scenarios, it is NO_VERSION_FILE, 
 NOT_IN_META_OR_DEPLOYED, NOT_IN_META, SHOULD_NOT_BE_DEPLOYED, 
 FIRST_REGION_STARTKEY_NOT_EMPTY, HOLE_IN_REGION_CHAIN.
 

 Key: HBASE-5599
 URL: https://issues.apache.org/jira/browse/HBASE-5599
 Project: HBase
  Issue Type: New Feature
  Components: hbck
Affects Versions: 0.90.6
Reporter: fulin wang
 Fix For: 0.90.6

 Attachments: hbase-5599-0.90.patch, hbase-5599-0.90_v2.patch, 
 hbase-5599-0.90_v3.patch, hbase-5599-0.90_v5.patch, hbase-5599-0.90_v6.patch, 
 hbase-5599-0.92_v5.patch, hbase-5599-0.94_v5.patch, hbase-5599-trunk_v5.patch


 The hbck tool can not fix the six scenarios.
 1. Version file does not exist in root dir.
Fix: I try to create a version file by 'FSUtils.setVersion' method.

 2. [REGIONNAME][KEY] on HDFS, but not listed in META or deployed on any 
 region server.
Fix: I get region info form the hdfs file, this region info write to 
 '.META.' table.

 3. [REGIONNAME][KEY] not in META, but deployed on [SERVERNAME]
Fix: I get region info form the hdfs file, this region info write to 
 '.META.' table.

 4. [REGIONNAME] should not be deployed according to META, but is deployed on 
 [SERVERNAME]
Fix: Close this region.

 5. First region should start with an empty key.  You need to  create a new 
 region and regioninfo in HDFS to plug the hole.
Fix: The region info is not in hdfs and .META., so it create a empty 
 region for this error.
 6. There is a hole in the region chain between [KEY] and [KEY]. You need to 
 create a new regioninfo and region dir in hdfs to plug the hole.
   Fix: The region info is not in hdfs and .META., so it create a empty region 
 for this hole.
   

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5674) add support in HBase to overwrite hbase timestamp to a version number during major compaction

2012-03-30 Thread He Yongqiang (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242115#comment-13242115
 ] 

He Yongqiang commented on HBASE-5674:
-

okay. Now i need to make it public on my lack sense of humor. :)

Here is the real problem:
In our use case, the space the data occupies *really* matter. We need to find 
all kind of things that we can do to bring down the size as much as possible. 
Apparently we do not want to bring in LZMA compression or bzip2 compression as 
they are really slow. In my simple test, a 41MB data can be reduced to 32MB 
after i rewrite the hbase Long timestamp to zero. The 8-bytes Long timestamp is 
heavy is because it is binary system timestamp which makes it very hard to 
compress (MemstoreTS is also a Long timestamp but there is no problem with it 
as it will be zero eventually). And if you look at how we are using that data, 
pretty much that data is not used by most applications if the data is system 
generated (not specified by applications). A good reason to make it 
configurable is some application may do specify it. In that case, pretty much 
you as hbase can not modify that data. But for a lot of other applications 
which do not care this data should not suffer this problem if data size really 
matter to them. 
I think this could benefit other community members as they may see this problem 
when they want to decrease the data size. 



 add support in HBase to overwrite hbase timestamp to a version number during 
 major compaction
 -

 Key: HBASE-5674
 URL: https://issues.apache.org/jira/browse/HBASE-5674
 Project: HBase
  Issue Type: Improvement
Reporter: He Yongqiang
Assignee: He Yongqiang

 Right now, a millisecond-level timestamp is attached to every record. 
 In our case, we only need a version number (mostly it will be just zero etc). 
 A millisecond timestamp is too heavy to carry. We should add support to 
 overwrite it to zero during major compaction. 
 KVs before major compaction will remain using system timestamp. And this 
 should be configurable, so that we should not mess up if the hbase timestamp 
 is specified by application.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5573) Replace client ZooKeeper watchers by simple ZooKeeper reads

2012-03-30 Thread nkeywal (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242117#comment-13242117
 ] 

nkeywal commented on HBASE-5573:


It can be committed imho. 

 Replace client ZooKeeper watchers by simple ZooKeeper reads
 ---

 Key: HBASE-5573
 URL: https://issues.apache.org/jira/browse/HBASE-5573
 Project: HBase
  Issue Type: Improvement
  Components: client, zookeeper
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Attachments: 5573.v1.patch, 5573.v2.patch, 5573.v4.patch, 
 5573.v6.patch, 5573.v7.patch, 5573.v8.patch


 Some code in the package needs to read data in ZK. This could be done by a 
 simple read, but is actually implemented with a watcher. This holds ZK 
 resources.
 Fixing this could also be an opportunity to remove the need for the client to 
 provide the master address and port.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5677) The master never does balance because duplicate openhandled the one region

2012-03-30 Thread xufeng (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242118#comment-13242118
 ] 

xufeng commented on HBASE-5677:
---

If region be assigned When the master is doing initialization(before do 
processFailover),the region will be duplicate openhandled.
because the unassigned node in zookeeper will be handled again in 
AssignmentManager#processFailover()

I use the 0.90 vsersion.
I found this issue in my cluster.

1.The system did not do balance:
{noformat}
Not running balancer because 2 region(s) in transition: 
{f4ff609df50e5bc9049fe202bb90f22e=hbase0205test,0038613850505050,1333033465665.f4ff609df50e5bc9049fe202bb90f22e.
 
state=OPEN, ts=1333036748502, 
febe5bb42ec841f7a9086d3b7bf0637c=hbase0205test,0038613802020202,1333033465474.febe5bb42ec841f7a9086d3b7bf0637c...
{noformat}

2.Choose f4ff609df50e5bc9049fe202bb90f22e as a simple to track.

3.In master log I found:
logA:
{noformat}
Line 17884: [2012-03-29 15:05:08,082] [DEBUG] 
[MASTER_OPEN_REGION-158-1-130-18:2-1] 
[org.apache.hadoop.hbase.master.handler.OpenedRegionHandler 138] The master has 
opened the region 
hbase0205test,0038613850505050,1333033465665.f4ff609df50e5bc9049fe202bb90f22e. 
that was online on serverName=158-1-130-18,20020,1332952904731, 
load=(requests=, regions=728, usedHeap=141, maxHeap=8165)
{noformat}

logB:
{noformat}
=Line 17885: [2012-03-29 15:05:08,082] [DEBUG] [master-158-1-130-18:2] 
[org.apache.hadoop.hbase.master.handler.OpenedRegionHandler 138] Handling 
OPENED event for 
hbase0205test,0038613850505050,1333033465665.f4ff609df50e5bc9049fe202bb90f22e. 
from serverName=158-1-130-18,20020,1332952904731, load=(requests=245, 
regions=758, usedHeap=145, maxHeap=8165); deleting unassigned node
Line 17897: [2012-03-29 15:05:08,084] [DEBUG] [master-158-1-130-18:2] 
[org.apache.hadoop.hbase.zookeeper.ZKAssign 511] master:2-0x236552a09e20353 
Deleting existing unassigned node for f4ff609df50e5bc9049fe202bb90f22e that is 
in expected state RS_ZK_REGION_OPENED
Line 17898: [2012-03-29 15:05:08,092] [WARN ] [master-158-1-130-18:2] 
[org.apache.hadoop.hbase.master.handler.OpenedRegionHandler 123] The znode of 
the region 
hbase0205test,0038613850505050,1333033465665.f4ff609df50e5bc9049fe202bb90f22e. 
would have already been deleted
Line 17899: [2012-03-29 15:05:08,092] [ERROR] [master-158-1-130-18:2] 
[org.apache.hadoop.hbase.master.handler.OpenedRegionHandler 97] The znode of 
region 
hbase0205test,0038613850505050,1333033465665.f4ff609df50e5bc9049fe202bb90f22e. 
could not be deleted.
{noformat}

4.The logA and logB should not appear at the same time,because belong to the 
same code in the region open flow.

5.So I ensure that this region has been handled duplicate.

6.Those log can explain what I write in Description:
Enable the table:
{noformat}
Line 16925: [2012-03-29 15:04:59,875] [DEBUG] 
[158-1-130-18:2-org.apache.hadoop.hbase.master.handler.EnableTableHandler$BulkEnabler-0]
 [org.apache.hadoop.hbase.zookeeper.ZKAssign 289] 
master:2-0x236552a09e20353 Creating (or updating) unassigned node for 
f4ff609df50e5bc9049fe202bb90f22e with OFFLINE state
{noformat}

Failover:
{noformat}
[2012-03-29 15:05:00,906] [INFO ] [master-158-1-130-18:2] 
[org.apache.hadoop.hbase.master.AssignmentManager 284] Failed-over master needs 
to process 66 regions in transition
{noformat}

 The master never does balance because duplicate openhandled the one region
 --

 Key: HBASE-5677
 URL: https://issues.apache.org/jira/browse/HBASE-5677
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90
Reporter: xufeng
Assignee: xufeng

 If region be assigned When the master is doing initialization(before do 
 processFailover),the region will be duplicate openhandled.
 it cause the region in RIT,thus the master never does balance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5677) The master never does balance because duplicate openhandled the one region

2012-03-30 Thread xufeng (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

xufeng updated HBASE-5677:
--

Description: 
If region be assigned When the master is doing initialization(before do 
processFailover),the region will be duplicate openhandled.
because the unassigned node in zookeeper will be handled again in 
AssignmentManager#processFailover()
it cause the region in RIT,thus the master never does balance.




  was:
If region be assigned When the master is doing initialization(before do 
processFailover),the region will be duplicate openhandled.
it cause the region in RIT,thus the master never does balance.


 The master never does balance because duplicate openhandled the one region
 --

 Key: HBASE-5677
 URL: https://issues.apache.org/jira/browse/HBASE-5677
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90
Reporter: xufeng
Assignee: xufeng

 If region be assigned When the master is doing initialization(before do 
 processFailover),the region will be duplicate openhandled.
 because the unassigned node in zookeeper will be handled again in 
 AssignmentManager#processFailover()
 it cause the region in RIT,thus the master never does balance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-4254) Get tests passing on Hadoop 23

2012-03-30 Thread Gregory Chanan (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-4254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242122#comment-13242122
 ] 

Gregory Chanan commented on HBASE-4254:
---

I did some playing around with TestLogRolling.testLogRollOnPipelineRestart 
against 0.23.  It looks like calling recoverLease on the HLog path and waiting 
awhile before trying to create the reader works.  If you don't call 
recoverLease the reader will never succeed.

Let's assume, for the sake of argument, that replicas waiting recovery will not 
return a full visible length.

1) Are we supposed to have to call recoverLease? (I'm guessing the recovery 
design doc covers this, I'll check it out)
2) Where should we call recoverLease?  Just in the test?  In HLog.createWriter? 
(it would block for awhile if we actually waited for the recovery)?

 Get tests passing on Hadoop 23
 --

 Key: HBASE-4254
 URL: https://issues.apache.org/jira/browse/HBASE-4254
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
 Fix For: 0.92.2


 Currently some 30 or so tests are failing on the HBase-trunk-on-hadoop-23 
 build. It looks like most are reflection-based issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HBASE-5678) Dynamic configuration capability for Hbase.

2012-03-30 Thread Uma Maheswara Rao G (Created) (JIRA)
Dynamic configuration capability for Hbase.
---

 Key: HBASE-5678
 URL: https://issues.apache.org/jira/browse/HBASE-5678
 Project: HBase
  Issue Type: New Feature
  Components: master, regionserver, security
Affects Versions: 0.96.0
Reporter: Uma Maheswara Rao G
Assignee: Uma Maheswara Rao G
 Fix For: 0.96.0


I think, some preperties can be danamically configured without restart of the 
nodes.
This is an umberilla JIRA for this Feature.

   In Hadoop we already had such feature but not yet implemented by nodes. I 
think we can have the similar base framework here and can implemented by nodes. 
So, that whatever properies are allowed to reconfigurable, should be able to 
reconfigure with new values with out restarting the node.

I will come up with some design doc with noeds implementation and will raise 
subtasks for each.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HBASE-5679) [89-fb] Port load test tool and related unit tests from trunk to 89-fb

2012-03-30 Thread Mikhail Bautin (Created) (JIRA)
[89-fb] Port load test tool and related unit tests from trunk to 89-fb
--

 Key: HBASE-5679
 URL: https://issues.apache.org/jira/browse/HBASE-5679
 Project: HBase
  Issue Type: Test
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin


When open-sourcing LoadTestTool that originated in 89-fb, numerous improvements 
to the tool were made, and unit tests based on the tool were created as part of 
HBASE-4908. These improvements need to be ported back to 89-fb.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5344) [89-fb] Scan unassigned region directory on master failover

2012-03-30 Thread Phabricator (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5344:
---

Attachment: D1605.2.patch

mbautin updated the revision [jira] [HBASE-5344] [89-fb] Scan unassigned 
region directory on master failover.
Reviewers: Kannan, Karthik, Liyin, JIRA, stack

  Deleting a bunch of files that should be deleted in this patch and fixing a 
compilation error in TestMiniClusterLoadSequential.

REVISION DETAIL
  https://reviews.facebook.net/D1605

AFFECTED FILES
  pom.xml
  src/main/java/org/apache/hadoop/hbase/EmptyWatcher.java
  src/test/java/org/apache/hadoop/hbase/EmptyWatcher.java
  src/main/java/org/apache/hadoop/hbase/HConstants.java
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
  src/main/java/org/apache/hadoop/hbase/util/AbstractHBaseTool.java
  src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
  src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
  src/test/java/org/apache/hadoop/hbase/manual/HBaseTest.java
  src/test/java/org/apache/hadoop/hbase/manual/RestartMetaTest.java
  src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java
  src/test/java/org/apache/hadoop/hbase/manual/utils/KillProcessesAndVerify.java
  src/test/java/org/apache/hadoop/hbase/manual/utils/MultiThreadedAction.java
  src/test/java/org/apache/hadoop/hbase/manual/utils/MultiThreadedReader.java
  src/test/java/org/apache/hadoop/hbase/manual/utils/MultiThreadedWriter.java
  
src/test/java/org/apache/hadoop/hbase/manual/utils/ProcessBasedLocalHBaseCluster.java
  src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java
  src/test/java/org/apache/hadoop/hbase/util/MultiThreadedAction.java
  src/test/java/org/apache/hadoop/hbase/util/MultiThreadedReader.java
  src/test/java/org/apache/hadoop/hbase/util/MultiThreadedWriter.java
  src/test/java/org/apache/hadoop/hbase/util/ProcessBasedLocalHBaseCluster.java
  src/test/java/org/apache/hadoop/hbase/util/RestartMetaTest.java
  src/test/java/org/apache/hadoop/hbase/util/TestLoadTestKVGenerator.java
  src/test/java/org/apache/hadoop/hbase/util/TestMiniClusterLoadEncoded.java
  src/test/java/org/apache/hadoop/hbase/util/TestMiniClusterLoadParallel.java
  src/test/java/org/apache/hadoop/hbase/util/TestMiniClusterLoadSequential.java


 [89-fb] Scan unassigned region directory on master failover
 ---

 Key: HBASE-5344
 URL: https://issues.apache.org/jira/browse/HBASE-5344
 Project: HBase
  Issue Type: Bug
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin
 Attachments: D1605.1.patch, D1605.2.patch


 In case the master dies after a regionserver writes region state as OPENED or 
 CLOSED in ZK but before the update is received by master and written to meta, 
 the new master that comes up has to pick up the region state from ZK and 
 write it to meta. Otherwise we can get multiply-assigned regions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5344) [89-fb] Scan unassigned region directory on master failover

2012-03-30 Thread Phabricator (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5344:
---

Attachment: D1605.3.patch

mbautin updated the revision [jira] [HBASE-5344] [89-fb] Scan unassigned 
region directory on master failover.
Reviewers: Kannan, Karthik, Liyin, JIRA, stack

  Attached the wrong diff to this revision. Restoring its old state rebased on 
recent changes.

REVISION DETAIL
  https://reviews.facebook.net/D1605

AFFECTED FILES
  src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionEventData.java
  src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java
  src/main/java/org/apache/hadoop/hbase/master/BaseScanner.java
  src/main/java/org/apache/hadoop/hbase/master/DirectRegionServerScanner.java
  src/main/java/org/apache/hadoop/hbase/master/HMaster.java
  src/main/java/org/apache/hadoop/hbase/master/ProcessRegionOpen.java
  src/main/java/org/apache/hadoop/hbase/master/RegionManager.java
  src/main/java/org/apache/hadoop/hbase/master/RootScanner.java
  src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
  src/main/java/org/apache/hadoop/hbase/master/ZKUnassignedWatcher.java
  
src/main/java/org/apache/hadoop/hbase/master/handler/MasterOpenRegionHandler.java
  
src/test/java/org/apache/hadoop/hbase/master/TestRegionStateOnMasterFailure.java


 [89-fb] Scan unassigned region directory on master failover
 ---

 Key: HBASE-5344
 URL: https://issues.apache.org/jira/browse/HBASE-5344
 Project: HBase
  Issue Type: Bug
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin
 Attachments: D1605.1.patch, D1605.2.patch, D1605.3.patch


 In case the master dies after a regionserver writes region state as OPENED or 
 CLOSED in ZK but before the update is received by master and written to meta, 
 the new master that comes up has to pick up the region state from ZK and 
 write it to meta. Otherwise we can get multiply-assigned regions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5535) Make the functions in task monitor synchronized

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242160#comment-13242160
 ] 

Hudson commented on HBASE-5535:
---

Integrated in HBase-0.92 #346 (See 
[https://builds.apache.org/job/HBase-0.92/346/])
HBASE-5535 Make the functions in task monitor synchronized (Revision 
1307281)

 Result = FAILURE
stack : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/monitoring/TaskMonitor.java


 Make the functions in task monitor synchronized
 ---

 Key: HBASE-5535
 URL: https://issues.apache.org/jira/browse/HBASE-5535
 Project: HBase
  Issue Type: Bug
Reporter: Liyin Tang
Assignee: Liyin Tang
 Fix For: 0.92.2, 0.94.0

 Attachments: 
 HBASE-5535-Make-the-functions-in-task-monitor-synchr-2012-03-08_16_33_42.patch


 There are some potential race condition in the task monitor. So update the 
 functions in task monitor to be synchronized.
 The example of the problem caused by the race condition:
 ERROR org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Cache flush 
 failed for region 
 java.lang.IndexOutOfBoundsException: Index: 1745, Size: 1744
 at java.util.ArrayList.add(ArrayList.java:367)
 at java.util.SubList.add(AbstractList.java:633)
 at java.util.SubList.add(AbstractList.java:633)
 at java.util.SubList.add(AbstractList.java:633)
 at java.util.SubList.add(AbstractList.java:633)
 at java.util.SubList.add(AbstractList.java:633)
 at java.util.AbstractList.add(AbstractList.java:91)
 at 
 org.apache.hadoop.hbase.monitoring.TaskMonitor.createStatus(TaskMonitor.java:74)
 at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1139)
 at 
 org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:260)
 at 
 org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:234)
 at 
 org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:146)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5673) The OOM problem of IPC client call cause all handle block

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242161#comment-13242161
 ] 

Hudson commented on HBASE-5673:
---

Integrated in HBase-0.92 #346 (See 
[https://builds.apache.org/job/HBase-0.92/346/])
HBASE-5673 The OOM problem of IPC client call cause all handle block; 
REAPPLY; V2 OF PATCH (Revision 1307275)
HBASE-5673 The OOM problem of IPC client call cause all handle block; REVERT -- 
APPLIED WRONG PATCH (Revision 1307272)

 Result = FAILURE
stack : 
Files : 
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java

stack : 
Files : 
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java


 The OOM problem of IPC client call  cause all handle block
 --

 Key: HBASE-5673
 URL: https://issues.apache.org/jira/browse/HBASE-5673
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90.6
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2, 0.94.1

 Attachments: HBASE-5673-90-V2.patch, HBASE-5673-90.patch


 if HBaseClient meet unable to create new native thread exception, the call 
 will never complete because it be lost in calls queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5678) Dynamic configuration capability for Hbase.

2012-03-30 Thread Uma Maheswara Rao G (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uma Maheswara Rao G updated HBASE-5678:
---

Component/s: (was: security)
 util

 Dynamic configuration capability for Hbase.
 ---

 Key: HBASE-5678
 URL: https://issues.apache.org/jira/browse/HBASE-5678
 Project: HBase
  Issue Type: New Feature
  Components: master, regionserver, util
Affects Versions: 0.96.0
Reporter: Uma Maheswara Rao G
Assignee: Uma Maheswara Rao G
 Fix For: 0.96.0


 I think, some preperties can be danamically configured without restart of the 
 nodes.
 This is an umberilla JIRA for this Feature.
In Hadoop we already had such feature but not yet implemented by nodes. I 
 think we can have the similar base framework here and can implemented by 
 nodes. So, that whatever properies are allowed to reconfigurable, should be 
 able to reconfigure with new values with out restarting the node.
 I will come up with some design doc with noeds implementation and will raise 
 subtasks for each.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5673) The OOM problem of IPC client call cause all handle block

2012-03-30 Thread xufeng (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242175#comment-13242175
 ] 

xufeng commented on HBASE-5673:
---

Build failed!
My patch cause it happened?


 The OOM problem of IPC client call  cause all handle block
 --

 Key: HBASE-5673
 URL: https://issues.apache.org/jira/browse/HBASE-5673
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90.6
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2, 0.94.1

 Attachments: HBASE-5673-90-V2.patch, HBASE-5673-90.patch


 if HBaseClient meet unable to create new native thread exception, the call 
 will never complete because it be lost in calls queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5564) Bulkload is discarding duplicate records

2012-03-30 Thread Laxman (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242189#comment-13242189
 ] 

Laxman commented on HBASE-5564:
---

Thanks for the commit stack.

 Bulkload is discarding duplicate records
 

 Key: HBASE-5564
 URL: https://issues.apache.org/jira/browse/HBASE-5564
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 0.96.0
 Environment: HBase 0.92
Reporter: Laxman
Assignee: Laxman
  Labels: bulkloader
 Fix For: 0.96.0

 Attachments: 5564.lint, 5564v5.txt, HBASE-5564_trunk.1.patch, 
 HBASE-5564_trunk.1.patch, HBASE-5564_trunk.2.patch, HBASE-5564_trunk.3.patch, 
 HBASE-5564_trunk.4_final.patch, HBASE-5564_trunk.patch


 Duplicate records are getting discarded when duplicate records exists in same 
 input file and more specifically if they exists in same split.
 Duplicate records are considered if the records are from diffrent different 
 splits.
 Version under test: HBase 0.92

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HBASE-5680) Hbase94 and Hbase 92.2 is not compatible with the Hadoop 23.1

2012-03-30 Thread Kristam Subba Swathi (Created) (JIRA)
Hbase94 and Hbase 92.2 is not compatible with the Hadoop 23.1 
--

 Key: HBASE-5680
 URL: https://issues.apache.org/jira/browse/HBASE-5680
 Project: HBase
  Issue Type: Bug
  Components: master
Reporter: Kristam Subba Swathi


Hmaster is not able to start because of the following error
Please find the following error 

2012-03-30 11:12:19,487 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled 
exception. Starting shutdown.
java.lang.NoClassDefFoundError: 
org/apache/hadoop/hdfs/protocol/FSConstants$SafeModeAction
at org.apache.hadoop.hbase.util.FSUtils.waitOnSafeMode(FSUtils.java:524)
at 
org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:324)
at 
org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:127)
at 
org.apache.hadoop.hbase.master.MasterFileSystem.init(MasterFileSystem.java:112)
at 
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:496)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:363)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.ClassNotFoundException: 
org.apache.hadoop.hdfs.protocol.FSConstants$SafeModeAction
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 7 more
There is a change in the FSConstants


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5617) Provide coprocessor hooks in put flow while rollbackMemstore.

2012-03-30 Thread ramkrishna.s.vasudevan (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242231#comment-13242231
 ] 

ramkrishna.s.vasudevan commented on HBASE-5617:
---


bq. Check out HRegion.mutateRowsWithLocks in 0.94 and 
HRegion.processRowsWithLocks in 0.96.

These apis roll back the kvs directly.  In my patch i thought of having putList.

So can we now pass kvs?
Or can we pass ListMutation and leave the user to do a instanceof check and 
proceed?

Also is the name ok to have rollbackMemstore in it or only rollback?  Kindly 
let me know your opinion on this.  
Also in trunk the processor.postProcess is done even if there is a failure.  
But in 0.94 its not.


 Provide coprocessor hooks in put flow while rollbackMemstore.
 -

 Key: HBASE-5617
 URL: https://issues.apache.org/jira/browse/HBASE-5617
 Project: HBase
  Issue Type: Improvement
  Components: coprocessors
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.96.0

 Attachments: HBASE-5617_1.patch, HBASE-5617_2.patch


 With coprocessors hooks while put happens we have the provision to create new 
 puts to other tables or regions.  These puts can be done with writeToWal as 
 false.
 In 0.94 and above the puts are first written to memstore and then to WAL.  If 
 any failure in the WAL append or sync the memstore is rollbacked.  
 Now the problem is that if the put that happens in the main flow fails there 
 is no way to rollback the 
 puts that happened in the prePut.
 We can add coprocessor hooks to like pre/postRoolBackMemStore.  Is any one 
 hook enough here?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5680) Hbase94 and Hbase 92.2 is not compatible with the Hadoop 23.1

2012-03-30 Thread Uma Maheswara Rao G (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242330#comment-13242330
 ] 

Uma Maheswara Rao G commented on HBASE-5680:


I remember, in 23 Version FSConstants has been deprecated.

{code}
@Deprecated
public abstract class FSConstants extends HdfsConstants {
}
{code}

And also HdfsConstants marked as InterfaceAudience.Private

{code}
@InterfaceAudience.Private
public class HdfsConstants {
{code}

I just commented in HDFS-1599, accessing something internal to DFS will create 
difficulties in later times for migrating the versions with Hadoop right. no?



 Hbase94 and Hbase 92.2 is not compatible with the Hadoop 23.1 
 --

 Key: HBASE-5680
 URL: https://issues.apache.org/jira/browse/HBASE-5680
 Project: HBase
  Issue Type: Bug
  Components: master
Reporter: Kristam Subba Swathi

 Hmaster is not able to start because of the following error
 Please find the following error 
 
 2012-03-30 11:12:19,487 FATAL org.apache.hadoop.hbase.master.HMaster: 
 Unhandled exception. Starting shutdown.
 java.lang.NoClassDefFoundError: 
 org/apache/hadoop/hdfs/protocol/FSConstants$SafeModeAction
   at org.apache.hadoop.hbase.util.FSUtils.waitOnSafeMode(FSUtils.java:524)
   at 
 org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:324)
   at 
 org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:127)
   at 
 org.apache.hadoop.hbase.master.MasterFileSystem.init(MasterFileSystem.java:112)
   at 
 org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:496)
   at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:363)
   at java.lang.Thread.run(Thread.java:662)
 Caused by: java.lang.ClassNotFoundException: 
 org.apache.hadoop.hdfs.protocol.FSConstants$SafeModeAction
   at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
   at java.security.AccessController.doPrivileged(Native Method)
   at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
   ... 7 more
 There is a change in the FSConstants

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy

2012-03-30 Thread Zhihong Yu (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242394#comment-13242394
 ] 

Zhihong Yu commented on HBASE-5655:
---

Integrated to TRUNK.

Thanks for the patch Himanshu.

Thanks for the review Stack.

 Cap space usage of default log4j rolling policy
 ---

 Key: HBASE-5655
 URL: https://issues.apache.org/jira/browse/HBASE-5655
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.92.1
Reporter: Himanshu Vashishtha
Assignee: Himanshu Vashishtha
 Fix For: 0.96.0

 Attachments: 5655-v1.patch, HBase-5655-v2.patch, HBase-5655-v3.patch


 The current default log4j policy is to use Daily Rolling File Appender 
 (DRFA). At times, its good to have a cap on the maximum size of the logs in 
 order to limit its disk usage. Here is a proposal to set a new file appemder 
 (RFA) as the default appender. It can be configured via env so that existing 
 tools can use the current behavior of using DRFA instead. 
 This is in parallel with jira Hadoop-8149.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5617) Provide coprocessor hooks in put flow while rollbackMemstore.

2012-03-30 Thread Zhihong Yu (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242421#comment-13242421
 ] 

Zhihong Yu commented on HBASE-5617:
---

I think passing ListKeyValue is general.

rollbackMemstore would be a good name. See javadoc for Store.rollback().

 Provide coprocessor hooks in put flow while rollbackMemstore.
 -

 Key: HBASE-5617
 URL: https://issues.apache.org/jira/browse/HBASE-5617
 Project: HBase
  Issue Type: Improvement
  Components: coprocessors
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.96.0

 Attachments: HBASE-5617_1.patch, HBASE-5617_2.patch


 With coprocessors hooks while put happens we have the provision to create new 
 puts to other tables or regions.  These puts can be done with writeToWal as 
 false.
 In 0.94 and above the puts are first written to memstore and then to WAL.  If 
 any failure in the WAL append or sync the memstore is rollbacked.  
 Now the problem is that if the put that happens in the main flow fails there 
 is no way to rollback the 
 puts that happened in the prePut.
 We can add coprocessor hooks to like pre/postRoolBackMemStore.  Is any one 
 hook enough here?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5667) RegexStringComparator supports java.util.regex.Pattern flags

2012-03-30 Thread David Arthur (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Arthur updated HBASE-5667:


Attachment: HBASE-5667.diff

* Changed constructor to (String, int)
* Added example in docs
* Removed DOTALL from (String) constructor

 RegexStringComparator supports java.util.regex.Pattern flags
 

 Key: HBASE-5667
 URL: https://issues.apache.org/jira/browse/HBASE-5667
 Project: HBase
  Issue Type: Improvement
  Components: filters
Reporter: David Arthur
Priority: Minor
 Attachments: HBASE-5667.diff, HBASE-5667.diff


 * Add constructor that takes in a Pattern
 * Add Pattern's flags to Writable fields, and actually use them when 
 recomposing the Filter

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5671) hbase.metrics.showTableName should be true by default

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242444#comment-13242444
 ] 

Hudson commented on HBASE-5671:
---

Integrated in HBase-0.94-security #6 (See 
[https://builds.apache.org/job/HBase-0.94-security/6/])
HBASE-5671 hbase.metrics.showTableName should be true by default (Revision 
1306924)

 Result = SUCCESS
stack : 
Files : 
* /hbase/branches/0.94/src/main/resources/hbase-default.xml


 hbase.metrics.showTableName should be true by default
 -

 Key: HBASE-5671
 URL: https://issues.apache.org/jira/browse/HBASE-5671
 Project: HBase
  Issue Type: Improvement
  Components: metrics
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Priority: Critical
 Fix For: 0.96.0, 0.94.1

 Attachments: HBASE-5671_v1.patch


 HBASE-4768 added per-cf metrics and a new configuration option 
 hbase.metrics.showTableName. We should switch the conf option to true by 
 default, since it is not intuitive (at least to me) to aggregate per-cf 
 across tables by default, and it seems confusing to report on cf's without 
 table names. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5633) NPE reading ZK config in HBase

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242445#comment-13242445
 ] 

Hudson commented on HBASE-5633:
---

Integrated in HBase-0.94-security #6 (See 
[https://builds.apache.org/job/HBase-0.94-security/6/])
HBASE-5638 Readability improvements on HBASE-5633: NPE reading ZK config in 
HBase (Matteo Bertozzi) (Revision 1307086)

 Result = SUCCESS
jmhsieh : 
Files : 
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HConstants.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKConfig.java


 NPE reading ZK config in HBase
 --

 Key: HBASE-5633
 URL: https://issues.apache.org/jira/browse/HBASE-5633
 Project: HBase
  Issue Type: Bug
  Components: zookeeper
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
 Fix For: 0.94.0

 Attachments: HBASE-5633-0.90.patch, HBASE-5633-0.92.patch, 
 HBASE-5633-v1.patch, HBASE-5633-v2.patch


 If zoo.cfg contains server.* (server.0=server0:2888:3888\n) and 
 cluster.distributed property (in hbase-site.xml) is empty we get an NPE in 
 parseZooCfg().
 The easy way to reproduce the bug is running 
 org.apache.hbase.zookeeper.TestHQuorumPeer with hbase-site.xml containing:
 {code}
 property
   namehbase.cluster.distributed/name
   value/value
 /property
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5670) Have Mutation implement the Row interface.

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242446#comment-13242446
 ] 

Hudson commented on HBASE-5670:
---

Integrated in HBase-0.94-security #6 (See 
[https://builds.apache.org/job/HBase-0.94-security/6/])
HBASE-5670 Have Mutation implement the Row interface. (Revision 1306958)

 Result = SUCCESS
larsh : 
Files : 
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/Append.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/Delete.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HTable.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/Mutation.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/Put.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java


 Have Mutation implement the Row interface.
 --

 Key: HBASE-5670
 URL: https://issues.apache.org/jira/browse/HBASE-5670
 Project: HBase
  Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Trivial
 Fix For: 0.96.0, 0.94.1

 Attachments: 5670-0.94.txt, 5670-trunk.txt, 5670-trunk.txt


 In HBASE-4347 I factored some code from Put/Delete/Append in Mutation.
 In a discussion with a co-worker I noticed that Put/Delete/Append still 
 implement the Row interface, but Mutation does not.
 In a trivial change I would like to move that interface up to Mutation, along 
 with changing HTable.batch(ListRow) to HTable.batch(List? extends Row) 
 (HConnection.processBatch takes List? extends Row already anyway), so that 
 HTable.batch can be used with a list of Mutations.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5638) Backport to 0.90 and 0.92 - NPE reading ZK config in HBase

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242448#comment-13242448
 ] 

Hudson commented on HBASE-5638:
---

Integrated in HBase-0.94-security #6 (See 
[https://builds.apache.org/job/HBase-0.94-security/6/])
HBASE-5638 Readability improvements on HBASE-5633: NPE reading ZK config in 
HBase (Matteo Bertozzi) (Revision 1307086)

 Result = SUCCESS
jmhsieh : 
Files : 
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HConstants.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKConfig.java


 Backport to 0.90 and 0.92 - NPE reading ZK config in HBase
 --

 Key: HBASE-5638
 URL: https://issues.apache.org/jira/browse/HBASE-5638
 Project: HBase
  Issue Type: Sub-task
  Components: zookeeper
Affects Versions: 0.90.6, 0.92.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.94.1

 Attachments: HBASE-5633-0.90.patch, HBASE-5633-0.92.patch, 
 HBASE-5638-0.90-v1.patch, HBASE-5638-0.90-v2.patch, HBASE-5638-0.92-v1.patch, 
 HBASE-5638-0.92-v2.patch, HBASE-5638-trunk-v1.patch, HBASE-5638-trunk-v2.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-4398) If HRegionPartitioner is used in MapReduce, client side configurations are overwritten by hbase-site.xml.

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-4398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242447#comment-13242447
 ] 

Hudson commented on HBASE-4398:
---

Integrated in HBase-0.94-security #6 (See 
[https://builds.apache.org/job/HBase-0.94-security/6/])
HBASE-4398 If HRegionPartitioner is used in MapReduce, client side 
configurations are overwritten by hbase-site.xml (Revision 1307117)

 Result = SUCCESS
stack : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/HRegionPartitioner.java


 If HRegionPartitioner is used in MapReduce, client side configurations are 
 overwritten by hbase-site.xml.
 -

 Key: HBASE-4398
 URL: https://issues.apache.org/jira/browse/HBASE-4398
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 0.90.4
Reporter: Takuya Ueshin
Assignee: Takuya Ueshin
 Fix For: 0.92.2, 0.94.1

 Attachments: HBASE-4398.patch


 If HRegionPartitioner is used in MapReduce, client side configurations are 
 overwritten by hbase-site.xml.
 We can reproduce the problem by the following instructions:
 {noformat}
 - Add HRegionPartitioner.class to the 4th argument of
 TableMapReduceUtil#initTableReducerJob()
 at line around 133
 in src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java
 - Change or remove hbase.zookeeper.property.clientPort property
 in hbase-site.xml ( for example, changed to 12345 ).
 - run testMultiRegionTable()
 {noformat}
 Then I got error messages as following:
 {noformat}
 2011-09-12 22:28:51,020 DEBUG [Thread-832] zookeeper.ZKUtil(93): hconnection 
 opening connection to ZooKeeper with ensemble (localhost:12345)
 2011-09-12 22:28:51,022 INFO  [Thread-832] 
 zookeeper.RecoverableZooKeeper(89): The identifier of this process is 
 43200@imac.local
 2011-09-12 22:28:51,123 WARN  [Thread-832] 
 zookeeper.RecoverableZooKeeper(161): Possibly transient ZooKeeper exception: 
 org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode 
 = ConnectionLoss for /hbase/master
 2011-09-12 22:28:51,123 INFO  [Thread-832] 
 zookeeper.RecoverableZooKeeper(173): The 1 times to retry ZooKeeper after 
 sleeping 1000 ms
  =
 2011-09-12 22:29:02,418 ERROR [Thread-832] mapreduce.HRegionPartitioner(125): 
 java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2e54e48d
  closed
 2011-09-12 22:29:02,422 WARN  [Thread-832] mapred.LocalJobRunner$Job(256): 
 job_local_0001
 java.lang.NullPointerException
at 
 org.apache.hadoop.hbase.mapreduce.HRegionPartitioner.setConf(HRegionPartitioner.java:128)
at 
 org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
at 
 org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at 
 org.apache.hadoop.mapred.MapTask$NewOutputCollector.init(MapTask.java:527)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at 
 org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
 {noformat}
 I think HTable should connect to ZooKeeper at port 21818 configured at client 
 side instead of 12345 in hbase-site.xml
 and It might be caused by HBaseConfiguration.addHbaseResources(conf); in 
 HRegionPartitioner#setConf(Configuration).
 And this might mean that all of client side configurations, also configured 
 in hbase-site.xml, are overwritten caused by this problem.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5097) RegionObserver implementation whose preScannerOpen and postScannerOpen Impl return null can stall the system initialization through NPE

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242450#comment-13242450
 ] 

Hudson commented on HBASE-5097:
---

Integrated in HBase-0.94-security #6 (See 
[https://builds.apache.org/job/HBase-0.94-security/6/])
HBASE-5097 RegionObserver implementation whose preScannerOpen and 
postScannerOpen Impl return null can stall the system initialization through 
NPE (Ram) (Revision 1307035)

 Result = SUCCESS
ramkrishna : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java


 RegionObserver implementation whose preScannerOpen and postScannerOpen Impl 
 return null can stall the system initialization through NPE
 ---

 Key: HBASE-5097
 URL: https://issues.apache.org/jira/browse/HBASE-5097
 Project: HBase
  Issue Type: Bug
  Components: coprocessors
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.92.2, 0.96.0, 0.94.1

 Attachments: HBASE-5097.patch, HBASE-5097_1.patch, HBASE-5097_2.patch


 In HRegionServer.java openScanner()
 {code}
   r.prepareScanner(scan);
   RegionScanner s = null;
   if (r.getCoprocessorHost() != null) {
 s = r.getCoprocessorHost().preScannerOpen(scan);
   }
   if (s == null) {
 s = r.getScanner(scan);
   }
   if (r.getCoprocessorHost() != null) {
 s = r.getCoprocessorHost().postScannerOpen(scan, s);
   }
 {code}
 If we dont have implemention for postScannerOpen the RegionScanner is null 
 and so throwing nullpointer 
 {code}
 java.lang.NullPointerException
   at 
 java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:881)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.addScanner(HRegionServer.java:2282)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2272)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
   at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
 {code}
 Making this defect as blocker.. Pls feel free to change the priority if am 
 wrong.  Also correct me if my way of trying out coprocessors without 
 implementing postScannerOpen is wrong.  Am just a learner.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5673) The OOM problem of IPC client call cause all handle block

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242449#comment-13242449
 ] 

Hudson commented on HBASE-5673:
---

Integrated in HBase-0.94-security #6 (See 
[https://builds.apache.org/job/HBase-0.94-security/6/])
HBASE-5673 The OOM problem of IPC client call cause all handle block; 
REAPPLY; V2 OF PATCH (Revision 1307276)
HBASE-5673 The OOM problem of IPC client call cause all handle block; REVERT -- 
APPLIED WRONG PATCH (Revision 1307271)
HBASE-5673 The OOM problem of IPC client call cause all handle block (Revision 
1307242)

 Result = SUCCESS
stack : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java

stack : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java

stack : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java


 The OOM problem of IPC client call  cause all handle block
 --

 Key: HBASE-5673
 URL: https://issues.apache.org/jira/browse/HBASE-5673
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90.6
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2, 0.94.1

 Attachments: HBASE-5673-90-V2.patch, HBASE-5673-90.patch


 if HBaseClient meet unable to create new native thread exception, the call 
 will never complete because it be lost in calls queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5674) add support in HBase to overwrite hbase timestamp to a version number during major compaction

2012-03-30 Thread Matt Corgan (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242460#comment-13242460
 ] 

Matt Corgan commented on HBASE-5674:


I've been brainstorming something similar as a follow-on to HBASE-4676.  The 
more similar timestamps you have in a block, the smaller the encoded version.  
Most people doing a simple, flat table with 1 version of each cell don't care 
about the timestamps.  They're only needed to pick the latest cell.  If all 
timestamps in an HFile are the same then they will encode down to nothing.

One possibility is to have an option flattenTimestamps where you grab 
t=currentTimeMillis() at the beginning of a flush and overwrite all timestamps 
with it.  To support multiple versions of a cell, you could use t-1, t-2, etc 
(as long as they don't go all the way back to the previous hfile's timestamp).

 add support in HBase to overwrite hbase timestamp to a version number during 
 major compaction
 -

 Key: HBASE-5674
 URL: https://issues.apache.org/jira/browse/HBASE-5674
 Project: HBase
  Issue Type: Improvement
Reporter: He Yongqiang
Assignee: He Yongqiang

 Right now, a millisecond-level timestamp is attached to every record. 
 In our case, we only need a version number (mostly it will be just zero etc). 
 A millisecond timestamp is too heavy to carry. We should add support to 
 overwrite it to zero during major compaction. 
 KVs before major compaction will remain using system timestamp. And this 
 should be configurable, so that we should not mess up if the hbase timestamp 
 is specified by application.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5670) Have Mutation implement the Row interface.

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242472#comment-13242472
 ] 

Hudson commented on HBASE-5670:
---

Integrated in HBase-TRUNK #2698 (See 
[https://builds.apache.org/job/HBase-TRUNK/2698/])
HBASE-5670 Have Mutation implement the Row interface. (Revision 1306959)

 Result = FAILURE
larsh : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Append.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Delete.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Mutation.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Put.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java


 Have Mutation implement the Row interface.
 --

 Key: HBASE-5670
 URL: https://issues.apache.org/jira/browse/HBASE-5670
 Project: HBase
  Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Trivial
 Fix For: 0.96.0, 0.94.1

 Attachments: 5670-0.94.txt, 5670-trunk.txt, 5670-trunk.txt


 In HBASE-4347 I factored some code from Put/Delete/Append in Mutation.
 In a discussion with a co-worker I noticed that Put/Delete/Append still 
 implement the Row interface, but Mutation does not.
 In a trivial change I would like to move that interface up to Mutation, along 
 with changing HTable.batch(ListRow) to HTable.batch(List? extends Row) 
 (HConnection.processBatch takes List? extends Row already anyway), so that 
 HTable.batch can be used with a list of Mutations.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5638) Backport to 0.90 and 0.92 - NPE reading ZK config in HBase

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242474#comment-13242474
 ] 

Hudson commented on HBASE-5638:
---

Integrated in HBase-TRUNK #2698 (See 
[https://builds.apache.org/job/HBase-TRUNK/2698/])
HBASE-5638 Readability improvements on HBASE-5633: NPE reading ZK config in 
HBase (Matteo Bertozzi) (Revision 1307085)

 Result = FAILURE
jmhsieh : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HConstants.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKConfig.java


 Backport to 0.90 and 0.92 - NPE reading ZK config in HBase
 --

 Key: HBASE-5638
 URL: https://issues.apache.org/jira/browse/HBASE-5638
 Project: HBase
  Issue Type: Sub-task
  Components: zookeeper
Affects Versions: 0.90.6, 0.92.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.94.1

 Attachments: HBASE-5633-0.90.patch, HBASE-5633-0.92.patch, 
 HBASE-5638-0.90-v1.patch, HBASE-5638-0.90-v2.patch, HBASE-5638-0.92-v1.patch, 
 HBASE-5638-0.92-v2.patch, HBASE-5638-trunk-v1.patch, HBASE-5638-trunk-v2.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5633) NPE reading ZK config in HBase

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242471#comment-13242471
 ] 

Hudson commented on HBASE-5633:
---

Integrated in HBase-TRUNK #2698 (See 
[https://builds.apache.org/job/HBase-TRUNK/2698/])
HBASE-5638 Readability improvements on HBASE-5633: NPE reading ZK config in 
HBase (Matteo Bertozzi) (Revision 1307085)

 Result = FAILURE
jmhsieh : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HConstants.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKConfig.java


 NPE reading ZK config in HBase
 --

 Key: HBASE-5633
 URL: https://issues.apache.org/jira/browse/HBASE-5633
 Project: HBase
  Issue Type: Bug
  Components: zookeeper
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
 Fix For: 0.94.0

 Attachments: HBASE-5633-0.90.patch, HBASE-5633-0.92.patch, 
 HBASE-5633-v1.patch, HBASE-5633-v2.patch


 If zoo.cfg contains server.* (server.0=server0:2888:3888\n) and 
 cluster.distributed property (in hbase-site.xml) is empty we get an NPE in 
 parseZooCfg().
 The easy way to reproduce the bug is running 
 org.apache.hbase.zookeeper.TestHQuorumPeer with hbase-site.xml containing:
 {code}
 property
   namehbase.cluster.distributed/name
   value/value
 /property
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5671) hbase.metrics.showTableName should be true by default

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242470#comment-13242470
 ] 

Hudson commented on HBASE-5671:
---

Integrated in HBase-TRUNK #2698 (See 
[https://builds.apache.org/job/HBase-TRUNK/2698/])
HBASE-5671 hbase.metrics.showTableName should be true by default (Revision 
1306712)

 Result = FAILURE
stack : 
Files : 
* /hbase/trunk/src/main/resources/hbase-default.xml


 hbase.metrics.showTableName should be true by default
 -

 Key: HBASE-5671
 URL: https://issues.apache.org/jira/browse/HBASE-5671
 Project: HBase
  Issue Type: Improvement
  Components: metrics
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Priority: Critical
 Fix For: 0.96.0, 0.94.1

 Attachments: HBASE-5671_v1.patch


 HBASE-4768 added per-cf metrics and a new configuration option 
 hbase.metrics.showTableName. We should switch the conf option to true by 
 default, since it is not intuitive (at least to me) to aggregate per-cf 
 across tables by default, and it seems confusing to report on cf's without 
 table names. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5564) Bulkload is discarding duplicate records

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242477#comment-13242477
 ] 

Hudson commented on HBASE-5564:
---

Integrated in HBase-TRUNK #2698 (See 
[https://builds.apache.org/job/HBase-TRUNK/2698/])
HBASE-5564 Bulkload is discarding duplicate records (Revision 1306907)

 Result = FAILURE
stack : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/ImportTsv.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/TsvImporterMapper.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java


 Bulkload is discarding duplicate records
 

 Key: HBASE-5564
 URL: https://issues.apache.org/jira/browse/HBASE-5564
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 0.96.0
 Environment: HBase 0.92
Reporter: Laxman
Assignee: Laxman
  Labels: bulkloader
 Fix For: 0.96.0

 Attachments: 5564.lint, 5564v5.txt, HBASE-5564_trunk.1.patch, 
 HBASE-5564_trunk.1.patch, HBASE-5564_trunk.2.patch, HBASE-5564_trunk.3.patch, 
 HBASE-5564_trunk.4_final.patch, HBASE-5564_trunk.patch


 Duplicate records are getting discarded when duplicate records exists in same 
 input file and more specifically if they exists in same split.
 Duplicate records are considered if the records are from diffrent different 
 splits.
 Version under test: HBase 0.92

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5655) Cap space usage of default log4j rolling policy

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242476#comment-13242476
 ] 

Hudson commented on HBASE-5655:
---

Integrated in HBase-TRUNK #2698 (See 
[https://builds.apache.org/job/HBase-TRUNK/2698/])
HBASE-5655 Cap space usage of default log4j rolling policy (Himanshu) 
(Revision 1307453)

 Result = FAILURE
tedyu : 
Files : 
* /hbase/trunk/bin/hbase-daemon.sh
* /hbase/trunk/conf/hbase-env.sh
* /hbase/trunk/conf/log4j.properties


 Cap space usage of default log4j rolling policy
 ---

 Key: HBASE-5655
 URL: https://issues.apache.org/jira/browse/HBASE-5655
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.92.1
Reporter: Himanshu Vashishtha
Assignee: Himanshu Vashishtha
 Fix For: 0.96.0

 Attachments: 5655-v1.patch, HBase-5655-v2.patch, HBase-5655-v3.patch


 The current default log4j policy is to use Daily Rolling File Appender 
 (DRFA). At times, its good to have a cap on the maximum size of the logs in 
 order to limit its disk usage. Here is a proposal to set a new file appemder 
 (RFA) as the default appender. It can be configured via env so that existing 
 tools can use the current behavior of using DRFA instead. 
 This is in parallel with jira Hadoop-8149.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5673) The OOM problem of IPC client call cause all handle block

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242475#comment-13242475
 ] 

Hudson commented on HBASE-5673:
---

Integrated in HBase-TRUNK #2698 (See 
[https://builds.apache.org/job/HBase-TRUNK/2698/])
HBASE-5673 The OOM problem of IPC client call cause all handle block; 
REAPPLY; V2 OF PATCH (Revision 1307277)
HBASE-5673 The OOM problem of IPC client call cause all handle block; REVERT -- 
APPLIED WRONG PATCH (Revision 1307270)
HBASE-5673 The OOM problem of IPC client call cause all handle block (Revision 
1307240)

 Result = FAILURE
stack : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java

stack : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java

stack : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java


 The OOM problem of IPC client call  cause all handle block
 --

 Key: HBASE-5673
 URL: https://issues.apache.org/jira/browse/HBASE-5673
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90.6
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2, 0.94.1

 Attachments: HBASE-5673-90-V2.patch, HBASE-5673-90.patch


 if HBaseClient meet unable to create new native thread exception, the call 
 will never complete because it be lost in calls queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5097) RegionObserver implementation whose preScannerOpen and postScannerOpen Impl return null can stall the system initialization through NPE

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242478#comment-13242478
 ] 

Hudson commented on HBASE-5097:
---

Integrated in HBase-TRUNK #2698 (See 
[https://builds.apache.org/job/HBase-TRUNK/2698/])
HBASE-5097 RegionObserver implementation whose preScannerOpen and 
postScannerOpen Impl return null can stall the system initialization through 
NPE (Ram) (Revision 1307036)

 Result = FAILURE
ramkrishna : 
Files : 
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java


 RegionObserver implementation whose preScannerOpen and postScannerOpen Impl 
 return null can stall the system initialization through NPE
 ---

 Key: HBASE-5097
 URL: https://issues.apache.org/jira/browse/HBASE-5097
 Project: HBase
  Issue Type: Bug
  Components: coprocessors
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.92.2, 0.96.0, 0.94.1

 Attachments: HBASE-5097.patch, HBASE-5097_1.patch, HBASE-5097_2.patch


 In HRegionServer.java openScanner()
 {code}
   r.prepareScanner(scan);
   RegionScanner s = null;
   if (r.getCoprocessorHost() != null) {
 s = r.getCoprocessorHost().preScannerOpen(scan);
   }
   if (s == null) {
 s = r.getScanner(scan);
   }
   if (r.getCoprocessorHost() != null) {
 s = r.getCoprocessorHost().postScannerOpen(scan, s);
   }
 {code}
 If we dont have implemention for postScannerOpen the RegionScanner is null 
 and so throwing nullpointer 
 {code}
 java.lang.NullPointerException
   at 
 java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:881)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.addScanner(HRegionServer.java:2282)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2272)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
   at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
 {code}
 Making this defect as blocker.. Pls feel free to change the priority if am 
 wrong.  Also correct me if my way of trying out coprocessors without 
 implementing postScannerOpen is wrong.  Am just a learner.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5673) The OOM problem of IPC client call cause all handle block

2012-03-30 Thread Zhihong Yu (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242495#comment-13242495
 ] 

Zhihong Yu commented on HBASE-5673:
---

I saw the following in jstack when running TestMultiVersions:
{code}
main prio=10 tid=0x40cc7800 nid=0x2b36 waiting on condition 
[0x7fd703f95000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at 
org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:211)
at 
org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:424)
at 
org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:200)
at 
org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:80)
at 
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:638)
at 
org.apache.hadoop.hbase.TestMultiVersions.testGetRowVersions(TestMultiVersions.java:143)
{code}

 The OOM problem of IPC client call  cause all handle block
 --

 Key: HBASE-5673
 URL: https://issues.apache.org/jira/browse/HBASE-5673
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90.6
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2, 0.94.1

 Attachments: HBASE-5673-90-V2.patch, HBASE-5673-90.patch


 if HBaseClient meet unable to create new native thread exception, the call 
 will never complete because it be lost in calls queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Reopened] (HBASE-5673) The OOM problem of IPC client call cause all handle block

2012-03-30 Thread Zhihong Yu (Reopened) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu reopened HBASE-5673:
---


The patch applies cleanly to TRUNK.

It was imprudent to integrate the patch without going through QA cycle.

Now all branches are broken.

 The OOM problem of IPC client call  cause all handle block
 --

 Key: HBASE-5673
 URL: https://issues.apache.org/jira/browse/HBASE-5673
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90.6
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2, 0.94.1

 Attachments: HBASE-5673-90-V2.patch, HBASE-5673-90.patch


 if HBaseClient meet unable to create new native thread exception, the call 
 will never complete because it be lost in calls queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5673) The OOM problem of IPC client call cause all handle block

2012-03-30 Thread Zhihong Yu (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242500#comment-13242500
 ] 

Zhihong Yu commented on HBASE-5673:
---

With the patch reverted, the test finished quickly:
{code}
Running org.apache.hadoop.hbase.TestMultiVersions
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.566 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] --- maven-surefire-plugin:2.10:test (secondPartTestsExecution) @ hbase 
---
[INFO] Tests are skipped.
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 52.152s
{code}

 The OOM problem of IPC client call  cause all handle block
 --

 Key: HBASE-5673
 URL: https://issues.apache.org/jira/browse/HBASE-5673
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90.6
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2, 0.94.1

 Attachments: HBASE-5673-90-V2.patch, HBASE-5673-90.patch


 if HBaseClient meet unable to create new native thread exception, the call 
 will never complete because it be lost in calls queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5667) RegexStringComparator supports java.util.regex.Pattern flags

2012-03-30 Thread Zhihong Yu (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242508#comment-13242508
 ] 

Zhihong Yu commented on HBASE-5667:
---

The new test failures are most likely inherited from TRUNK which were caused by 
HBASE-5673

 RegexStringComparator supports java.util.regex.Pattern flags
 

 Key: HBASE-5667
 URL: https://issues.apache.org/jira/browse/HBASE-5667
 Project: HBase
  Issue Type: Improvement
  Components: filters
Reporter: David Arthur
Priority: Minor
 Attachments: HBASE-5667.diff, HBASE-5667.diff


 * Add constructor that takes in a Pattern
 * Add Pattern's flags to Writable fields, and actually use them when 
 recomposing the Filter

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5674) add support in HBase to overwrite hbase timestamp to a version number during major compaction

2012-03-30 Thread stack (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242505#comment-13242505
 ] 

stack commented on HBASE-5674:
--

@He np.  Thanks for the background.  On a slightly related note, I was going to 
ask if you'd been following Matt's work over in hbase-4676.  The compression 
factor he gets over there I thought you'd be interested in.

 add support in HBase to overwrite hbase timestamp to a version number during 
 major compaction
 -

 Key: HBASE-5674
 URL: https://issues.apache.org/jira/browse/HBASE-5674
 Project: HBase
  Issue Type: Improvement
Reporter: He Yongqiang
Assignee: He Yongqiang

 Right now, a millisecond-level timestamp is attached to every record. 
 In our case, we only need a version number (mostly it will be just zero etc). 
 A millisecond timestamp is too heavy to carry. We should add support to 
 overwrite it to zero during major compaction. 
 KVs before major compaction will remain using system timestamp. And this 
 should be configurable, so that we should not mess up if the hbase timestamp 
 is specified by application.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5673) The OOM problem of IPC client call cause all handle block

2012-03-30 Thread stack (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242511#comment-13242511
 ] 

stack commented on HBASE-5673:
--

Np.  Will undo the commit

 The OOM problem of IPC client call  cause all handle block
 --

 Key: HBASE-5673
 URL: https://issues.apache.org/jira/browse/HBASE-5673
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90.6
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2, 0.94.1

 Attachments: HBASE-5673-90-V2.patch, HBASE-5673-90.patch


 if HBaseClient meet unable to create new native thread exception, the call 
 will never complete because it be lost in calls queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5673) The OOM problem of IPC client call cause all handle block

2012-03-30 Thread stack (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242514#comment-13242514
 ] 

stack commented on HBASE-5673:
--

Reverted everywhere.

 The OOM problem of IPC client call  cause all handle block
 --

 Key: HBASE-5673
 URL: https://issues.apache.org/jira/browse/HBASE-5673
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90.6
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2, 0.94.1

 Attachments: HBASE-5673-90-V2.patch, HBASE-5673-90.patch


 if HBaseClient meet unable to create new native thread exception, the call 
 will never complete because it be lost in calls queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5667) RegexStringComparator supports java.util.regex.Pattern flags

2012-03-30 Thread Zhihong Yu (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242521#comment-13242521
 ] 

Zhihong Yu commented on HBASE-5667:
---

I think we should keep the Pattern.DOTALL in (String) constructor:
{code}
-this.pattern = Pattern.compile(expr, Pattern.DOTALL);
+this.pattern = Pattern.compile(expr);
{code}

 RegexStringComparator supports java.util.regex.Pattern flags
 

 Key: HBASE-5667
 URL: https://issues.apache.org/jira/browse/HBASE-5667
 Project: HBase
  Issue Type: Improvement
  Components: filters
Reporter: David Arthur
Priority: Minor
 Attachments: HBASE-5667.diff, HBASE-5667.diff


 * Add constructor that takes in a Pattern
 * Add Pattern's flags to Writable fields, and actually use them when 
 recomposing the Filter

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5673) The OOM problem of IPC client call cause all handle block

2012-03-30 Thread stack (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242526#comment-13242526
 ] 

stack commented on HBASE-5673:
--

bq. It was imprudent to integrate the patch without going through QA cycle.

I got a little carried-away...  It happens.


@xufeng Can you figure why the test fail?  Test is probably doing silly 
depdendency on returned exception.

Also, was thinking later that you perhaps should check the Throwable.  If its 
already an IOE, don't wrap it in a new IOE?

Good stuff.

 The OOM problem of IPC client call  cause all handle block
 --

 Key: HBASE-5673
 URL: https://issues.apache.org/jira/browse/HBASE-5673
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90.6
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2, 0.94.1

 Attachments: HBASE-5673-90-V2.patch, HBASE-5673-90.patch


 if HBaseClient meet unable to create new native thread exception, the call 
 will never complete because it be lost in calls queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5655) Cap space usage of default log4j rolling policy

2012-03-30 Thread Himanshu Vashishtha (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Himanshu Vashishtha updated HBASE-5655:
---

Release Note: 
This changes the default log rolling scheme from DRFA to RFA. The former rolls 
over the log on a date change trigger, while the later rolls over when the log 
file size reaches a predefined limit. The issue with DRFA is that it doesn't 
have the ability to cap the space usage, so users who are not using host-level 
log rotation might fill up their log partitions. This results in a cluster 
crash. RFA puts a size limit on the log size and therefore is a safer option in 
such scenarios. The default file size is 256MB with 20 files (total of 5GB 
logs). In case one needs to revert to the original DRFA (for some legacy tools 
etc), one can set a env variable HBASE_ROOT_LOGGER to ROOT_LOGGER_LEVEL,DRFA. 
Please refer to the hbase-env.sh for more details.


 Cap space usage of default log4j rolling policy
 ---

 Key: HBASE-5655
 URL: https://issues.apache.org/jira/browse/HBASE-5655
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.92.1
Reporter: Himanshu Vashishtha
Assignee: Himanshu Vashishtha
 Fix For: 0.96.0

 Attachments: 5655-v1.patch, HBase-5655-v2.patch, HBase-5655-v3.patch


 The current default log4j policy is to use Daily Rolling File Appender 
 (DRFA). At times, its good to have a cap on the maximum size of the logs in 
 order to limit its disk usage. Here is a proposal to set a new file appemder 
 (RFA) as the default appender. It can be configured via env so that existing 
 tools can use the current behavior of using DRFA instead. 
 This is in parallel with jira Hadoop-8149.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5655) Cap space usage of default log4j rolling policy

2012-03-30 Thread Zhihong Yu (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5655:
--

Release Note: 
This changes the default log rolling scheme from DRFA to RFA. The former rolls 
over the log on a date change trigger, while the later rolls over when the log 
file size reaches a predefined limit. The issue with DRFA is that it doesn't 
have the ability to cap the space usage, so users who are not using host-level 
log rotation might fill up their log partitions. This results in a cluster 
crash. RFA puts a size limit on the log size and therefore is a safer option in 
such scenarios. The default file size is 256MB with 20 files (total of 5GB 
logs). In case one needs to revert to the original DRFA (for some legacy tools 
etc), one can set environment variable HBASE_ROOT_LOGGER to 
ROOT_LOGGER_LEVEL,DRFA. Please refer to the hbase-env.sh for more details.


  was:
This changes the default log rolling scheme from DRFA to RFA. The former rolls 
over the log on a date change trigger, while the later rolls over when the log 
file size reaches a predefined limit. The issue with DRFA is that it doesn't 
have the ability to cap the space usage, so users who are not using host-level 
log rotation might fill up their log partitions. This results in a cluster 
crash. RFA puts a size limit on the log size and therefore is a safer option in 
such scenarios. The default file size is 256MB with 20 files (total of 5GB 
logs). In case one needs to revert to the original DRFA (for some legacy tools 
etc), one can set a env variable HBASE_ROOT_LOGGER to ROOT_LOGGER_LEVEL,DRFA. 
Please refer to the hbase-env.sh for more details.



 Cap space usage of default log4j rolling policy
 ---

 Key: HBASE-5655
 URL: https://issues.apache.org/jira/browse/HBASE-5655
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.92.1
Reporter: Himanshu Vashishtha
Assignee: Himanshu Vashishtha
 Fix For: 0.96.0

 Attachments: 5655-v1.patch, HBase-5655-v2.patch, HBase-5655-v3.patch


 The current default log4j policy is to use Daily Rolling File Appender 
 (DRFA). At times, its good to have a cap on the maximum size of the logs in 
 order to limit its disk usage. Here is a proposal to set a new file appemder 
 (RFA) as the default appender. It can be configured via env so that existing 
 tools can use the current behavior of using DRFA instead. 
 This is in parallel with jira Hadoop-8149.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5667) RegexStringComparator supports java.util.regex.Pattern flags

2012-03-30 Thread stack (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242531#comment-13242531
 ] 

stack commented on HBASE-5667:
--

Yes, and call through to the other constructor so its super(expr, 
Pattern.DOTALL).

 RegexStringComparator supports java.util.regex.Pattern flags
 

 Key: HBASE-5667
 URL: https://issues.apache.org/jira/browse/HBASE-5667
 Project: HBase
  Issue Type: Improvement
  Components: filters
Reporter: David Arthur
Priority: Minor
 Attachments: HBASE-5667.diff, HBASE-5667.diff


 * Add constructor that takes in a Pattern
 * Add Pattern's flags to Writable fields, and actually use them when 
 recomposing the Filter

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5667) RegexStringComparator supports java.util.regex.Pattern flags

2012-03-30 Thread David Arthur (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Arthur updated HBASE-5667:


Attachment: HBASE-5667.diff

Yea on second thought keeping DOTALL is best for compatability. However, I 
don't like implicit things like that, so I added a comment to the (String) 
explaining it.

 RegexStringComparator supports java.util.regex.Pattern flags
 

 Key: HBASE-5667
 URL: https://issues.apache.org/jira/browse/HBASE-5667
 Project: HBase
  Issue Type: Improvement
  Components: filters
Reporter: David Arthur
Priority: Minor
 Attachments: HBASE-5667.diff, HBASE-5667.diff, HBASE-5667.diff


 * Add constructor that takes in a Pattern
 * Add Pattern's flags to Writable fields, and actually use them when 
 recomposing the Filter

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5667) RegexStringComparator supports java.util.regex.Pattern flags

2012-03-30 Thread Zhihong Yu (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5667:
--

Fix Version/s: 0.96.0
 Hadoop Flags: Reviewed

 RegexStringComparator supports java.util.regex.Pattern flags
 

 Key: HBASE-5667
 URL: https://issues.apache.org/jira/browse/HBASE-5667
 Project: HBase
  Issue Type: Improvement
  Components: filters
Reporter: David Arthur
Assignee: David Arthur
Priority: Minor
 Fix For: 0.96.0

 Attachments: HBASE-5667.diff, HBASE-5667.diff, HBASE-5667.diff


 * Add constructor that takes in a Pattern
 * Add Pattern's flags to Writable fields, and actually use them when 
 recomposing the Filter

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Assigned] (HBASE-5667) RegexStringComparator supports java.util.regex.Pattern flags

2012-03-30 Thread Zhihong Yu (Assigned) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu reassigned HBASE-5667:
-

Assignee: David Arthur

 RegexStringComparator supports java.util.regex.Pattern flags
 

 Key: HBASE-5667
 URL: https://issues.apache.org/jira/browse/HBASE-5667
 Project: HBase
  Issue Type: Improvement
  Components: filters
Reporter: David Arthur
Assignee: David Arthur
Priority: Minor
 Fix For: 0.96.0

 Attachments: HBASE-5667.diff, HBASE-5667.diff, HBASE-5667.diff


 * Add constructor that takes in a Pattern
 * Add Pattern's flags to Writable fields, and actually use them when 
 recomposing the Filter

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5667) RegexStringComparator supports java.util.regex.Pattern flags

2012-03-30 Thread Zhihong Yu (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5667:
--

Comment: was deleted

(was: -1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12520612/HBASE-5667.diff
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.master.TestRestartCluster
  org.apache.hadoop.hbase.client.TestMetaMigrationRemovingHTD
  
org.apache.hadoop.hbase.io.encoding.TestUpgradeFromHFileV1ToEncoding
  org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildHole
  org.apache.hadoop.hbase.catalog.TestCatalogTrackerOnCluster
  org.apache.hadoop.hbase.master.TestMasterFailover
  org.apache.hadoop.hbase.mapreduce.TestImportTsv
  org.apache.hadoop.hbase.TestMultiVersions
  
org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildOverlap
  org.apache.hadoop.hbase.mapred.TestTableMapReduce
  org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1352//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1352//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1352//console

This message is automatically generated.)

 RegexStringComparator supports java.util.regex.Pattern flags
 

 Key: HBASE-5667
 URL: https://issues.apache.org/jira/browse/HBASE-5667
 Project: HBase
  Issue Type: Improvement
  Components: filters
Reporter: David Arthur
Assignee: David Arthur
Priority: Minor
 Fix For: 0.96.0

 Attachments: HBASE-5667.diff, HBASE-5667.diff, HBASE-5667.diff


 * Add constructor that takes in a Pattern
 * Add Pattern's flags to Writable fields, and actually use them when 
 recomposing the Filter

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5604) HLog replay tool that generates HFiles for use by LoadIncrementalHFiles.

2012-03-30 Thread Lars Hofhansl (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242584#comment-13242584
 ] 

Lars Hofhansl commented on HBASE-5604:
--

Ok... I'll do a separate tool called WALPlayer. The Import rationale would be 
that indeed right now it can import HLog files. I added HFiles output to Import 
in HBASE-5440.
HDFS would probably not work, as these files would be copied around for 
archiving.
Changing the names of WALs is interesting. I'd be worried about side-effects to 
other tools.

Maybe it's not even an issue. A reader of the HLog can tell after looking at 
the first log entries whether it can ignore the rest of the file.

 HLog replay tool that generates HFiles for use by LoadIncrementalHFiles.
 

 Key: HBASE-5604
 URL: https://issues.apache.org/jira/browse/HBASE-5604
 Project: HBase
  Issue Type: New Feature
Reporter: Lars Hofhansl

 Just an idea I had. Might be useful for restore of a backup using the HLogs.
 This could an M/R (with a mapper per HLog file).
 The tool would get a timerange and a (set of) table(s). We'd pick the right 
 HLogs based on time before the M/R job is started and then have a mapper per 
 HLog file.
 The mapper would then go through the HLog, filter all WALEdits that didn't 
 fit into the time range or are not any of the tables and then uses 
 HFileOutputFormat to generate HFiles.
 Would need to indicate the splits we want, probably from a live table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5535) Make the functions in task monitor synchronized

2012-03-30 Thread Jonathan Hsieh (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242591#comment-13242591
 ] 

Jonathan Hsieh commented on HBASE-5535:
---

I looked at the code and this may have been an alternate solution to the also 
applied HBASE-4386.  This doesn't really hurt the code (synchronized is 
slightly larger now) so we can leave it as is.

 Make the functions in task monitor synchronized
 ---

 Key: HBASE-5535
 URL: https://issues.apache.org/jira/browse/HBASE-5535
 Project: HBase
  Issue Type: Bug
Reporter: Liyin Tang
Assignee: Liyin Tang
 Fix For: 0.92.2, 0.94.0

 Attachments: 
 HBASE-5535-Make-the-functions-in-task-monitor-synchr-2012-03-08_16_33_42.patch


 There are some potential race condition in the task monitor. So update the 
 functions in task monitor to be synchronized.
 The example of the problem caused by the race condition:
 ERROR org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Cache flush 
 failed for region 
 java.lang.IndexOutOfBoundsException: Index: 1745, Size: 1744
 at java.util.ArrayList.add(ArrayList.java:367)
 at java.util.SubList.add(AbstractList.java:633)
 at java.util.SubList.add(AbstractList.java:633)
 at java.util.SubList.add(AbstractList.java:633)
 at java.util.SubList.add(AbstractList.java:633)
 at java.util.SubList.add(AbstractList.java:633)
 at java.util.AbstractList.add(AbstractList.java:91)
 at 
 org.apache.hadoop.hbase.monitoring.TaskMonitor.createStatus(TaskMonitor.java:74)
 at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1139)
 at 
 org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:260)
 at 
 org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:234)
 at 
 org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:146)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5573) Replace client ZooKeeper watchers by simple ZooKeeper reads

2012-03-30 Thread Zhihong Yu (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242594#comment-13242594
 ] 

Zhihong Yu commented on HBASE-5573:
---

I ran TestMasterObserver with patch v8 and didn't see failure.

Integrated to trunk.

Thanks for the patch, N.

Thanks for the review, Stack.

 Replace client ZooKeeper watchers by simple ZooKeeper reads
 ---

 Key: HBASE-5573
 URL: https://issues.apache.org/jira/browse/HBASE-5573
 Project: HBase
  Issue Type: Improvement
  Components: client, zookeeper
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Attachments: 5573.v1.patch, 5573.v2.patch, 5573.v4.patch, 
 5573.v6.patch, 5573.v7.patch, 5573.v8.patch


 Some code in the package needs to read data in ZK. This could be done by a 
 simple read, but is actually implemented with a watcher. This holds ZK 
 resources.
 Fixing this could also be an opportunity to remove the need for the client to 
 provide the master address and port.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5635) If getTaskList() returns null splitlogWorker is down. It wont serve any requests.

2012-03-30 Thread Chinna Rao Lalam (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chinna Rao Lalam updated HBASE-5635:


Attachment: HBASE-5635.1.patch

 If getTaskList() returns null splitlogWorker is down. It wont serve any 
 requests. 
 --

 Key: HBASE-5635
 URL: https://issues.apache.org/jira/browse/HBASE-5635
 Project: HBase
  Issue Type: Bug
  Components: wal
Affects Versions: 0.92.1
Reporter: Kristam Subba Swathi
 Attachments: HBASE-5635.1.patch, HBASE-5635.patch


 During the hlog split operation if all the zookeepers are down ,then the 
 paths will be returned as null and the splitworker thread wil be exited
 Now this regionserver wil not be able to acquire any other tasks since the 
 splitworker thread is exited
 Please find the attached code for more details
 {code}
 private ListString getTaskList() {
 for (int i = 0; i  zkretries; i++) {
   try {
 return (ZKUtil.listChildrenAndWatchForNewChildren(this.watcher,
 this.watcher.splitLogZNode));
   } catch (KeeperException e) {
 LOG.warn(Could not get children of znode  +
 this.watcher.splitLogZNode, e);
 try {
   Thread.sleep(1000);
 } catch (InterruptedException e1) {
   LOG.warn(Interrupted while trying to get task list ..., e1);
   Thread.currentThread().interrupt();
   return null;
 }
   }
 }
 {code}
 in the org.apache.hadoop.hbase.regionserver.SplitLogWorker 
  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5673) The OOM problem of IPC client call cause all handle block

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242597#comment-13242597
 ] 

Hudson commented on HBASE-5673:
---

Integrated in HBase-0.94 #70 (See 
[https://builds.apache.org/job/HBase-0.94/70/])
HBASE-5673 The OOM problem of IPC client call cause all handle block; 
REVERT OF V2, A SECOND REVERT (Revision 1307515)

 Result = SUCCESS
stack : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java


 The OOM problem of IPC client call  cause all handle block
 --

 Key: HBASE-5673
 URL: https://issues.apache.org/jira/browse/HBASE-5673
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90.6
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2, 0.94.1

 Attachments: HBASE-5673-90-V2.patch, HBASE-5673-90.patch


 if HBaseClient meet unable to create new native thread exception, the call 
 will never complete because it be lost in calls queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5635) If getTaskList() returns null splitlogWorker is down. It wont serve any requests.

2012-03-30 Thread Chinna Rao Lalam (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242602#comment-13242602
 ] 

Chinna Rao Lalam commented on HBASE-5635:
-

Updated the patch with retry logic in getTaskList(). So if getTaskList() after 
retry if it returns null the splitLogWorker will be exited.

 If getTaskList() returns null splitlogWorker is down. It wont serve any 
 requests. 
 --

 Key: HBASE-5635
 URL: https://issues.apache.org/jira/browse/HBASE-5635
 Project: HBase
  Issue Type: Bug
  Components: wal
Affects Versions: 0.92.1
Reporter: Kristam Subba Swathi
 Attachments: HBASE-5635.1.patch, HBASE-5635.patch


 During the hlog split operation if all the zookeepers are down ,then the 
 paths will be returned as null and the splitworker thread wil be exited
 Now this regionserver wil not be able to acquire any other tasks since the 
 splitworker thread is exited
 Please find the attached code for more details
 {code}
 private ListString getTaskList() {
 for (int i = 0; i  zkretries; i++) {
   try {
 return (ZKUtil.listChildrenAndWatchForNewChildren(this.watcher,
 this.watcher.splitLogZNode));
   } catch (KeeperException e) {
 LOG.warn(Could not get children of znode  +
 this.watcher.splitLogZNode, e);
 try {
   Thread.sleep(1000);
 } catch (InterruptedException e1) {
   LOG.warn(Interrupted while trying to get task list ..., e1);
   Thread.currentThread().interrupt();
   return null;
 }
   }
 }
 {code}
 in the org.apache.hadoop.hbase.regionserver.SplitLogWorker 
  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5635) If getTaskList() returns null splitlogWorker is down. It wont serve any requests.

2012-03-30 Thread Zhihong Yu (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242607#comment-13242607
 ] 

Zhihong Yu commented on HBASE-5635:
---

{code}
+// It will be in loop till it get the list of children or
+// it will come out if worker thread existed.
{code}
Typo. First line should read 'it gets the'
Second line should read 'worker thread exited'.
{code}
+Thread.sleep(1000);
{code}
Please reduce the above interval.

Do we need a timeout for the newly added loop ?

 If getTaskList() returns null splitlogWorker is down. It wont serve any 
 requests. 
 --

 Key: HBASE-5635
 URL: https://issues.apache.org/jira/browse/HBASE-5635
 Project: HBase
  Issue Type: Bug
  Components: wal
Affects Versions: 0.92.1
Reporter: Kristam Subba Swathi
 Attachments: HBASE-5635.1.patch, HBASE-5635.patch


 During the hlog split operation if all the zookeepers are down ,then the 
 paths will be returned as null and the splitworker thread wil be exited
 Now this regionserver wil not be able to acquire any other tasks since the 
 splitworker thread is exited
 Please find the attached code for more details
 {code}
 private ListString getTaskList() {
 for (int i = 0; i  zkretries; i++) {
   try {
 return (ZKUtil.listChildrenAndWatchForNewChildren(this.watcher,
 this.watcher.splitLogZNode));
   } catch (KeeperException e) {
 LOG.warn(Could not get children of znode  +
 this.watcher.splitLogZNode, e);
 try {
   Thread.sleep(1000);
 } catch (InterruptedException e1) {
   LOG.warn(Interrupted while trying to get task list ..., e1);
   Thread.currentThread().interrupt();
   return null;
 }
   }
 }
 {code}
 in the org.apache.hadoop.hbase.regionserver.SplitLogWorker 
  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5667) RegexStringComparator supports java.util.regex.Pattern flags

2012-03-30 Thread Hadoop QA (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242613#comment-13242613
 ] 

Hadoop QA commented on HBASE-5667:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12520628/HBASE-5667.diff
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.mapreduce.TestImportTsv
  org.apache.hadoop.hbase.mapred.TestTableMapReduce
  org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1353//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1353//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1353//console

This message is automatically generated.

 RegexStringComparator supports java.util.regex.Pattern flags
 

 Key: HBASE-5667
 URL: https://issues.apache.org/jira/browse/HBASE-5667
 Project: HBase
  Issue Type: Improvement
  Components: filters
Reporter: David Arthur
Assignee: David Arthur
Priority: Minor
 Fix For: 0.96.0

 Attachments: HBASE-5667.diff, HBASE-5667.diff, HBASE-5667.diff


 * Add constructor that takes in a Pattern
 * Add Pattern's flags to Writable fields, and actually use them when 
 recomposing the Filter

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5667) RegexStringComparator supports java.util.regex.Pattern flags

2012-03-30 Thread Zhihong Yu (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242615#comment-13242615
 ] 

Zhihong Yu commented on HBASE-5667:
---

Patch v3 looks good.

Will integrate later if there is no objection.

 RegexStringComparator supports java.util.regex.Pattern flags
 

 Key: HBASE-5667
 URL: https://issues.apache.org/jira/browse/HBASE-5667
 Project: HBase
  Issue Type: Improvement
  Components: filters
Reporter: David Arthur
Assignee: David Arthur
Priority: Minor
 Fix For: 0.96.0

 Attachments: HBASE-5667.diff, HBASE-5667.diff, HBASE-5667.diff


 * Add constructor that takes in a Pattern
 * Add Pattern's flags to Writable fields, and actually use them when 
 recomposing the Filter

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5674) add support in HBase to overwrite hbase timestamp to a version number during major compaction

2012-03-30 Thread He Yongqiang (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242650#comment-13242650
 ] 

He Yongqiang commented on HBASE-5674:
-

Thanks Matt and stack for the point out of 4676. Yeah, we are very very 
interested in the work that is going on HBase-4767.

 add support in HBase to overwrite hbase timestamp to a version number during 
 major compaction
 -

 Key: HBASE-5674
 URL: https://issues.apache.org/jira/browse/HBASE-5674
 Project: HBase
  Issue Type: Improvement
Reporter: He Yongqiang
Assignee: He Yongqiang

 Right now, a millisecond-level timestamp is attached to every record. 
 In our case, we only need a version number (mostly it will be just zero etc). 
 A millisecond timestamp is too heavy to carry. We should add support to 
 overwrite it to zero during major compaction. 
 KVs before major compaction will remain using system timestamp. And this 
 should be configurable, so that we should not mess up if the hbase timestamp 
 is specified by application.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5673) The OOM problem of IPC client call cause all handle block

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242654#comment-13242654
 ] 

Hudson commented on HBASE-5673:
---

Integrated in HBase-0.92 #347 (See 
[https://builds.apache.org/job/HBase-0.92/347/])
HBASE-5673 The OOM problem of IPC client call cause all handle block; 
REVERT OF V2, A SECOND REVERT (Revision 1307516)

 Result = SUCCESS
stack : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java


 The OOM problem of IPC client call  cause all handle block
 --

 Key: HBASE-5673
 URL: https://issues.apache.org/jira/browse/HBASE-5673
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90.6
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2, 0.94.1

 Attachments: HBASE-5673-90-V2.patch, HBASE-5673-90.patch


 if HBaseClient meet unable to create new native thread exception, the call 
 will never complete because it be lost in calls queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HBASE-5681) Split Region crash if region is still offline after a previous split

2012-03-30 Thread Matteo Bertozzi (Created) (JIRA)
Split Region crash if region is still offline after a previous split


 Key: HBASE-5681
 URL: https://issues.apache.org/jira/browse/HBASE-5681
 Project: HBase
  Issue Type: Bug
  Components: regionserver, zookeeper
Affects Versions: 0.92.1, 0.96.0, 0.94.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor


I've a script that starts hbase and a couple of region servers in distributed 
mode (hbase.cluster.distributed = true) due to HBASE-5666 I need a sleep to 
ensure that rs are up.
{code}
$HBASE_HOME/bin/start-hbase.sh
sleep 5 # bug HBASE-5666 rs doesn't retry if znode is not available.
$HBASE_HOME/bin/local-regionservers.sh start 1 2 3
{code}

Once hbase is started I run an hbase shell script file (see below)
everything is fine till last split operation.
{code}
# $HBASE_HOME/bin/hbase shell test.hbase
# test.hbase
create 'bugtb-t1', 'tcf11', 'tcf12'
create 'bugtb-t2', 'tcf11', 'tcf12'

put 'bugtb-t1', '10', 'tcf11:c1', 'a'
put 'bugtb-t1', '15', 'tcf11:c2', 'b'
put 'bugtb-t1', '20', 'tcf11:c1', 'c'
put 'bugtb-t1', '30', 'tcf11:c2', 'd'
put 'bugtb-t1', '35', 'tcf11:c1', 'e'
put 'bugtb-t1', '40', 'tcf11:c2', 'f'

put 'bugtb-t2', '10', 'tcf11:c1', 'a'
put 'bugtb-t2', '15', 'tcf11:c2', 'b'
put 'bugtb-t2', '20', 'tcf11:c1', 'c'
put 'bugtb-t2', '30', 'tcf11:c2', 'd'
put 'bugtb-t2', '35', 'tcf11:c1', 'e'
put 'bugtb-t2', '40', 'tcf11:c2', 'f'

split 'bugtb-t1', '20'
split 'bugtb-t2', '20'
split 'bugtb-t1', '40'
{code}

During the last split the region is still offline, and you get an exception
(If you sleep a bit before executing the last split, everything is fine)
{code}
ERROR: org.apache.hadoop.ipc.RemoteException: 
org.apache.hadoop.hbase.NotServingRegionException: Region is not online: 
bugtb-t1,,1333134892936.4e14c2cf4293156d5b099dc3d5c44890.
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3123)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2926)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at 
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:366)
at 
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1383)
{code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5680) Hbase94 and Hbase 92.2 is not compatible with the Hadoop 23.1

2012-03-30 Thread Zhihong Yu (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242668#comment-13242668
 ] 

Zhihong Yu commented on HBASE-5680:
---

Using reflection should solve this problem.

 Hbase94 and Hbase 92.2 is not compatible with the Hadoop 23.1 
 --

 Key: HBASE-5680
 URL: https://issues.apache.org/jira/browse/HBASE-5680
 Project: HBase
  Issue Type: Bug
  Components: master
Reporter: Kristam Subba Swathi

 Hmaster is not able to start because of the following error
 Please find the following error 
 
 2012-03-30 11:12:19,487 FATAL org.apache.hadoop.hbase.master.HMaster: 
 Unhandled exception. Starting shutdown.
 java.lang.NoClassDefFoundError: 
 org/apache/hadoop/hdfs/protocol/FSConstants$SafeModeAction
   at org.apache.hadoop.hbase.util.FSUtils.waitOnSafeMode(FSUtils.java:524)
   at 
 org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:324)
   at 
 org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:127)
   at 
 org.apache.hadoop.hbase.master.MasterFileSystem.init(MasterFileSystem.java:112)
   at 
 org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:496)
   at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:363)
   at java.lang.Thread.run(Thread.java:662)
 Caused by: java.lang.ClassNotFoundException: 
 org.apache.hadoop.hdfs.protocol.FSConstants$SafeModeAction
   at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
   at java.security.AccessController.doPrivileged(Native Method)
   at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
   ... 7 more
 There is a change in the FSConstants

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5667) RegexStringComparator supports java.util.regex.Pattern flags

2012-03-30 Thread Zhihong Yu (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242672#comment-13242672
 ] 

Zhihong Yu commented on HBASE-5667:
---

Patch v3 integrated to TRUNK.

Thanks for the patch David.

Thanks for the review Stack.

 RegexStringComparator supports java.util.regex.Pattern flags
 

 Key: HBASE-5667
 URL: https://issues.apache.org/jira/browse/HBASE-5667
 Project: HBase
  Issue Type: Improvement
  Components: filters
Reporter: David Arthur
Assignee: David Arthur
Priority: Minor
 Fix For: 0.96.0

 Attachments: HBASE-5667.diff, HBASE-5667.diff, HBASE-5667.diff


 * Add constructor that takes in a Pattern
 * Add Pattern's flags to Writable fields, and actually use them when 
 recomposing the Filter

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Assigned] (HBASE-5666) RegionServer doesn't retry to check if base node is available

2012-03-30 Thread Matteo Bertozzi (Assigned) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi reassigned HBASE-5666:
--

Assignee: Matteo Bertozzi

 RegionServer doesn't retry to check if base node is available
 -

 Key: HBASE-5666
 URL: https://issues.apache.org/jira/browse/HBASE-5666
 Project: HBase
  Issue Type: Bug
  Components: regionserver, zookeeper
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Attachments: hbase-1-regionserver.log, hbase-2-regionserver.log, 
 hbase-3-regionserver.log, hbase-master.log, hbase-regionserver.log, 
 hbase-zookeeper.log


 I've a script that starts hbase and a couple of region servers in distributed 
 mode (hbase.cluster.distributed = true)
 {code}
 $HBASE_HOME/bin/start-hbase.sh
 $HBASE_HOME/bin/local-regionservers.sh start 1 2 3
 {code}
 but the region servers are not able to start...
 It seems that during the RS start the the znode is still not available, and 
 HRegionServer.initializeZooKeeper() check just once if the base not is 
 available.
 {code}
 2012-03-28 21:54:05,013 INFO 
 org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Check the value 
 configured in 'zookeeper.znode.parent'. There could be a mismatch with the 
 one configured in the master.
 2012-03-28 21:54:08,598 FATAL 
 org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server 
 localhost,60202,133296824: Initialization of RS failed.  Hence aborting 
 RS.
 java.io.IOException: Received the shutdown message while waiting.
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.blockAndCheckIfStopped(HRegionServer.java:626)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:596)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:558)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:672)
   at java.lang.Thread.run(Thread.java:662)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5681) Split Region crash if region is still offline after a previous split

2012-03-30 Thread Matteo Bertozzi (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-5681:
---

Attachment: logs1-HBASE-5681.tar.bz2
logs0-HBASE-5681.tar.bz2

 Split Region crash if region is still offline after a previous split
 

 Key: HBASE-5681
 URL: https://issues.apache.org/jira/browse/HBASE-5681
 Project: HBase
  Issue Type: Bug
  Components: regionserver, zookeeper
Affects Versions: 0.92.1, 0.96.0, 0.94.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
 Attachments: logs0-HBASE-5681.tar.bz2, logs1-HBASE-5681.tar.bz2


 I've a script that starts hbase and a couple of region servers in distributed 
 mode (hbase.cluster.distributed = true) due to HBASE-5666 I need a sleep to 
 ensure that rs are up.
 {code}
 $HBASE_HOME/bin/start-hbase.sh
 sleep 5 # bug HBASE-5666 rs doesn't retry if znode is not available.
 $HBASE_HOME/bin/local-regionservers.sh start 1 2 3
 {code}
 Once hbase is started I run an hbase shell script file (see below)
 everything is fine till last split operation.
 {code}
 # $HBASE_HOME/bin/hbase shell test.hbase
 # test.hbase
 create 'bugtb-t1', 'tcf11', 'tcf12'
 create 'bugtb-t2', 'tcf11', 'tcf12'
 put 'bugtb-t1', '10', 'tcf11:c1', 'a'
 put 'bugtb-t1', '15', 'tcf11:c2', 'b'
 put 'bugtb-t1', '20', 'tcf11:c1', 'c'
 put 'bugtb-t1', '30', 'tcf11:c2', 'd'
 put 'bugtb-t1', '35', 'tcf11:c1', 'e'
 put 'bugtb-t1', '40', 'tcf11:c2', 'f'
 put 'bugtb-t2', '10', 'tcf11:c1', 'a'
 put 'bugtb-t2', '15', 'tcf11:c2', 'b'
 put 'bugtb-t2', '20', 'tcf11:c1', 'c'
 put 'bugtb-t2', '30', 'tcf11:c2', 'd'
 put 'bugtb-t2', '35', 'tcf11:c1', 'e'
 put 'bugtb-t2', '40', 'tcf11:c2', 'f'
 split 'bugtb-t1', '20'
 split 'bugtb-t2', '20'
 split 'bugtb-t1', '40'
 {code}
 During the last split the region is still offline, and you get an 
 exception
 (If you sleep a bit before executing the last split, everything is fine)
 {code}
 ERROR: org.apache.hadoop.ipc.RemoteException: 
 org.apache.hadoop.hbase.NotServingRegionException: Region is not online: 
 bugtb-t1,,1333134892936.4e14c2cf4293156d5b099dc3d5c44890.
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3123)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2926)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:366)
   at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1383)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5666) RegionServer doesn't retry to check if base node is available

2012-03-30 Thread stack (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242684#comment-13242684
 ] 

stack commented on HBASE-5666:
--

We used to retry our very first zk operation a few times IIRC.

 RegionServer doesn't retry to check if base node is available
 -

 Key: HBASE-5666
 URL: https://issues.apache.org/jira/browse/HBASE-5666
 Project: HBase
  Issue Type: Bug
  Components: regionserver, zookeeper
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Attachments: hbase-1-regionserver.log, hbase-2-regionserver.log, 
 hbase-3-regionserver.log, hbase-master.log, hbase-regionserver.log, 
 hbase-zookeeper.log


 I've a script that starts hbase and a couple of region servers in distributed 
 mode (hbase.cluster.distributed = true)
 {code}
 $HBASE_HOME/bin/start-hbase.sh
 $HBASE_HOME/bin/local-regionservers.sh start 1 2 3
 {code}
 but the region servers are not able to start...
 It seems that during the RS start the the znode is still not available, and 
 HRegionServer.initializeZooKeeper() check just once if the base not is 
 available.
 {code}
 2012-03-28 21:54:05,013 INFO 
 org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Check the value 
 configured in 'zookeeper.znode.parent'. There could be a mismatch with the 
 one configured in the master.
 2012-03-28 21:54:08,598 FATAL 
 org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server 
 localhost,60202,133296824: Initialization of RS failed.  Hence aborting 
 RS.
 java.io.IOException: Received the shutdown message while waiting.
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.blockAndCheckIfStopped(HRegionServer.java:626)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:596)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:558)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:672)
   at java.lang.Thread.run(Thread.java:662)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5666) RegionServer doesn't retry to check if base node is available

2012-03-30 Thread Zhihong Yu (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242687#comment-13242687
 ] 

Zhihong Yu commented on HBASE-5666:
---

I think we can introduce something similar to the following:
{code}
  conf.getInt(hbase.catalogtracker.default.timeout, 1000));
{code}
We can call the new config parameter hbase.basenode.avail.timeout

 RegionServer doesn't retry to check if base node is available
 -

 Key: HBASE-5666
 URL: https://issues.apache.org/jira/browse/HBASE-5666
 Project: HBase
  Issue Type: Bug
  Components: regionserver, zookeeper
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Attachments: hbase-1-regionserver.log, hbase-2-regionserver.log, 
 hbase-3-regionserver.log, hbase-master.log, hbase-regionserver.log, 
 hbase-zookeeper.log


 I've a script that starts hbase and a couple of region servers in distributed 
 mode (hbase.cluster.distributed = true)
 {code}
 $HBASE_HOME/bin/start-hbase.sh
 $HBASE_HOME/bin/local-regionservers.sh start 1 2 3
 {code}
 but the region servers are not able to start...
 It seems that during the RS start the the znode is still not available, and 
 HRegionServer.initializeZooKeeper() check just once if the base not is 
 available.
 {code}
 2012-03-28 21:54:05,013 INFO 
 org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Check the value 
 configured in 'zookeeper.znode.parent'. There could be a mismatch with the 
 one configured in the master.
 2012-03-28 21:54:08,598 FATAL 
 org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server 
 localhost,60202,133296824: Initialization of RS failed.  Hence aborting 
 RS.
 java.io.IOException: Received the shutdown message while waiting.
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.blockAndCheckIfStopped(HRegionServer.java:626)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:596)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:558)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:672)
   at java.lang.Thread.run(Thread.java:662)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5666) RegionServer doesn't retry to check if base node is available

2012-03-30 Thread stack (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242696#comment-13242696
 ] 

stack commented on HBASE-5666:
--

Please be careful here.

What version of hbase are we talking of.

We used to retry the first zk operation and then recoverablezk took over 
thereafter.   Code in trunk was recently refactored as part of 'HBASE-5399 Cut 
the link between the client and the zookeeper ensemble'.  Is this a teething 
issue that comes of that commit?

 RegionServer doesn't retry to check if base node is available
 -

 Key: HBASE-5666
 URL: https://issues.apache.org/jira/browse/HBASE-5666
 Project: HBase
  Issue Type: Bug
  Components: regionserver, zookeeper
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Attachments: hbase-1-regionserver.log, hbase-2-regionserver.log, 
 hbase-3-regionserver.log, hbase-master.log, hbase-regionserver.log, 
 hbase-zookeeper.log


 I've a script that starts hbase and a couple of region servers in distributed 
 mode (hbase.cluster.distributed = true)
 {code}
 $HBASE_HOME/bin/start-hbase.sh
 $HBASE_HOME/bin/local-regionservers.sh start 1 2 3
 {code}
 but the region servers are not able to start...
 It seems that during the RS start the the znode is still not available, and 
 HRegionServer.initializeZooKeeper() check just once if the base not is 
 available.
 {code}
 2012-03-28 21:54:05,013 INFO 
 org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Check the value 
 configured in 'zookeeper.znode.parent'. There could be a mismatch with the 
 one configured in the master.
 2012-03-28 21:54:08,598 FATAL 
 org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server 
 localhost,60202,133296824: Initialization of RS failed.  Hence aborting 
 RS.
 java.io.IOException: Received the shutdown message while waiting.
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.blockAndCheckIfStopped(HRegionServer.java:626)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:596)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:558)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:672)
   at java.lang.Thread.run(Thread.java:662)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5681) Split Region crash if region is still offline after a previous split

2012-03-30 Thread stack (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242703#comment-13242703
 ] 

stack commented on HBASE-5681:
--

Is this the same as HBASE-5665?

 Split Region crash if region is still offline after a previous split
 

 Key: HBASE-5681
 URL: https://issues.apache.org/jira/browse/HBASE-5681
 Project: HBase
  Issue Type: Bug
  Components: regionserver, zookeeper
Affects Versions: 0.92.1, 0.96.0, 0.94.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
 Attachments: logs0-HBASE-5681.tar.bz2, logs1-HBASE-5681.tar.bz2


 I've a script that starts hbase and a couple of region servers in distributed 
 mode (hbase.cluster.distributed = true) due to HBASE-5666 I need a sleep to 
 ensure that rs are up.
 {code}
 $HBASE_HOME/bin/start-hbase.sh
 sleep 5 # bug HBASE-5666 rs doesn't retry if znode is not available.
 $HBASE_HOME/bin/local-regionservers.sh start 1 2 3
 {code}
 Once hbase is started I run an hbase shell script file (see below)
 everything is fine till last split operation.
 {code}
 # $HBASE_HOME/bin/hbase shell test.hbase
 # test.hbase
 create 'bugtb-t1', 'tcf11', 'tcf12'
 create 'bugtb-t2', 'tcf11', 'tcf12'
 put 'bugtb-t1', '10', 'tcf11:c1', 'a'
 put 'bugtb-t1', '15', 'tcf11:c2', 'b'
 put 'bugtb-t1', '20', 'tcf11:c1', 'c'
 put 'bugtb-t1', '30', 'tcf11:c2', 'd'
 put 'bugtb-t1', '35', 'tcf11:c1', 'e'
 put 'bugtb-t1', '40', 'tcf11:c2', 'f'
 put 'bugtb-t2', '10', 'tcf11:c1', 'a'
 put 'bugtb-t2', '15', 'tcf11:c2', 'b'
 put 'bugtb-t2', '20', 'tcf11:c1', 'c'
 put 'bugtb-t2', '30', 'tcf11:c2', 'd'
 put 'bugtb-t2', '35', 'tcf11:c1', 'e'
 put 'bugtb-t2', '40', 'tcf11:c2', 'f'
 split 'bugtb-t1', '20'
 split 'bugtb-t2', '20'
 split 'bugtb-t1', '40'
 {code}
 During the last split the region is still offline, and you get an 
 exception
 (If you sleep a bit before executing the last split, everything is fine)
 {code}
 ERROR: org.apache.hadoop.ipc.RemoteException: 
 org.apache.hadoop.hbase.NotServingRegionException: Region is not online: 
 bugtb-t1,,1333134892936.4e14c2cf4293156d5b099dc3d5c44890.
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3123)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2926)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:366)
   at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1383)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5673) The OOM problem of IPC client call cause all handle block

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242700#comment-13242700
 ] 

Hudson commented on HBASE-5673:
---

Integrated in HBase-TRUNK #2699 (See 
[https://builds.apache.org/job/HBase-TRUNK/2699/])
HBASE-5673 The OOM problem of IPC client call cause all handle block; 
REVERT OF V2, A SECOND REVERT (Revision 1307513)

 Result = FAILURE
stack : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java


 The OOM problem of IPC client call  cause all handle block
 --

 Key: HBASE-5673
 URL: https://issues.apache.org/jira/browse/HBASE-5673
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90.6
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2, 0.94.1

 Attachments: HBASE-5673-90-V2.patch, HBASE-5673-90.patch


 if HBaseClient meet unable to create new native thread exception, the call 
 will never complete because it be lost in calls queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242699#comment-13242699
 ] 

Hudson commented on HBASE-5542:
---

Integrated in HBase-TRUNK #2699 (See 
[https://builds.apache.org/job/HBase-TRUNK/2699/])
HBASE-5542 Addendum - accidentally checked in .orig file (Revision 1307562)

 Result = FAILURE
larsh : 
Files : 
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java.orig


 Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()
 

 Key: HBASE-5542
 URL: https://issues.apache.org/jira/browse/HBASE-5542
 Project: HBase
  Issue Type: Improvement
Reporter: Scott Chen
Assignee: Scott Chen
 Fix For: 0.96.0

 Attachments: HBASE-5542.2.txt, HBASE-5542.3.txt, HBASE-5542.4.txt, 
 HBASE-5542.4.txt, HBASE-5542.4.txt, HBASE-5542.D2217.1.patch, 
 HBASE-5542.D2217.10.patch, HBASE-5542.D2217.11.patch, 
 HBASE-5542.D2217.12.patch, HBASE-5542.D2217.13.patch, 
 HBASE-5542.D2217.14.patch, HBASE-5542.D2217.15.patch, 
 HBASE-5542.D2217.2.patch, HBASE-5542.D2217.3.patch, HBASE-5542.D2217.4.patch, 
 HBASE-5542.D2217.5.patch, HBASE-5542.D2217.6.patch, HBASE-5542.D2217.7.patch, 
 HBASE-5542.D2217.8.patch, HBASE-5542.D2217.9.patch, HBASE-5542.txt


 mutateRowsWithLocks() does atomic mutations on multiple rows.
 processRow() does atomic read-modify-writes on a single row.
 It will be useful to generalize both and have a
 processRowsWithLocks() that does atomic read-modify-writes on multiple rows.
 This also helps reduce some redundancy in the codes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5667) RegexStringComparator supports java.util.regex.Pattern flags

2012-03-30 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242701#comment-13242701
 ] 

Hudson commented on HBASE-5667:
---

Integrated in HBase-TRUNK #2699 (See 
[https://builds.apache.org/job/HBase-TRUNK/2699/])
HBASE-5667 RegexStringComparator supports java.util.regex.Pattern flags 
(David Arthur) (Revision 1307580)

 Result = FAILURE
tedyu : 
Files : 
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/filter/RegexStringComparator.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/filter/TestSingleColumnValueFilter.java


 RegexStringComparator supports java.util.regex.Pattern flags
 

 Key: HBASE-5667
 URL: https://issues.apache.org/jira/browse/HBASE-5667
 Project: HBase
  Issue Type: Improvement
  Components: filters
Reporter: David Arthur
Assignee: David Arthur
Priority: Minor
 Fix For: 0.96.0

 Attachments: HBASE-5667.diff, HBASE-5667.diff, HBASE-5667.diff


 * Add constructor that takes in a Pattern
 * Add Pattern's flags to Writable fields, and actually use them when 
 recomposing the Filter

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5619) Create PB protocols for HRegionInterface

2012-03-30 Thread Jimmy Xiang (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242720#comment-13242720
 ] 

Jimmy Xiang commented on HBASE-5619:


@Stack, thanks!

 Create PB protocols for HRegionInterface
 

 Key: HBASE-5619
 URL: https://issues.apache.org/jira/browse/HBASE-5619
 Project: HBase
  Issue Type: Sub-task
  Components: ipc, master, migration, regionserver
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.96.0

 Attachments: 5619v6.txt, 5619v6.txt, hbase-5619.patch, 
 hbase-5619_v3.patch, hbase-5619_v4.patch, hbase-5619_v5.patch


 Subtask of HBase-5443, separate HRegionInterface into admin protocol and 
 client protocol, create the PB protocol buffer files

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5619) Create PB protocols for HRegionInterface

2012-03-30 Thread stack (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5619:
-

Attachment: 5619v6.txt

Reattach to see if it triggers hadoopqa build

 Create PB protocols for HRegionInterface
 

 Key: HBASE-5619
 URL: https://issues.apache.org/jira/browse/HBASE-5619
 Project: HBase
  Issue Type: Sub-task
  Components: ipc, master, migration, regionserver
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.96.0

 Attachments: 5619v6.txt, 5619v6.txt, hbase-5619.patch, 
 hbase-5619_v3.patch, hbase-5619_v4.patch, hbase-5619_v5.patch


 Subtask of HBase-5443, separate HRegionInterface into admin protocol and 
 client protocol, create the PB protocol buffer files

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5084) Allow different HTable instances to share one ExecutorService

2012-03-30 Thread Lars Hofhansl (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5084:
-

Fix Version/s: 0.94.1

Going to work on this for 0.94.x.

 Allow different HTable instances to share one ExecutorService
 -

 Key: HBASE-5084
 URL: https://issues.apache.org/jira/browse/HBASE-5084
 Project: HBase
  Issue Type: Task
Reporter: Zhihong Yu
Assignee: Lars Hofhansl
 Fix For: 0.94.1


 This came out of Lily 1.1.1 release:
 Use a shared ExecutorService for all HTable instances, leading to better (or 
 actual) thread reuse

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5666) RegionServer doesn't retry to check if base node is available

2012-03-30 Thread Matteo Bertozzi (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242734#comment-13242734
 ] 

Matteo Bertozzi commented on HBASE-5666:


The exception seems to be present in 0.92 and trunk, but I've only looked at 
trunk code

 RegionServer doesn't retry to check if base node is available
 -

 Key: HBASE-5666
 URL: https://issues.apache.org/jira/browse/HBASE-5666
 Project: HBase
  Issue Type: Bug
  Components: regionserver, zookeeper
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Attachments: hbase-1-regionserver.log, hbase-2-regionserver.log, 
 hbase-3-regionserver.log, hbase-master.log, hbase-regionserver.log, 
 hbase-zookeeper.log


 I've a script that starts hbase and a couple of region servers in distributed 
 mode (hbase.cluster.distributed = true)
 {code}
 $HBASE_HOME/bin/start-hbase.sh
 $HBASE_HOME/bin/local-regionservers.sh start 1 2 3
 {code}
 but the region servers are not able to start...
 It seems that during the RS start the the znode is still not available, and 
 HRegionServer.initializeZooKeeper() check just once if the base not is 
 available.
 {code}
 2012-03-28 21:54:05,013 INFO 
 org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Check the value 
 configured in 'zookeeper.znode.parent'. There could be a mismatch with the 
 one configured in the master.
 2012-03-28 21:54:08,598 FATAL 
 org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server 
 localhost,60202,133296824: Initialization of RS failed.  Hence aborting 
 RS.
 java.io.IOException: Received the shutdown message while waiting.
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.blockAndCheckIfStopped(HRegionServer.java:626)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:596)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:558)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:672)
   at java.lang.Thread.run(Thread.java:662)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5681) Split Region crash if region is still offline after a previous split

2012-03-30 Thread Matteo Bertozzi (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242736#comment-13242736
 ] 

Matteo Bertozzi commented on HBASE-5681:


quite similar, i get both exception. The FileNotFound of HBASE-5665 and 
NotServingRegionException. I'll retry with 5665 patch.

 Split Region crash if region is still offline after a previous split
 

 Key: HBASE-5681
 URL: https://issues.apache.org/jira/browse/HBASE-5681
 Project: HBase
  Issue Type: Bug
  Components: regionserver, zookeeper
Affects Versions: 0.92.1, 0.96.0, 0.94.1
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
 Attachments: logs0-HBASE-5681.tar.bz2, logs1-HBASE-5681.tar.bz2


 I've a script that starts hbase and a couple of region servers in distributed 
 mode (hbase.cluster.distributed = true) due to HBASE-5666 I need a sleep to 
 ensure that rs are up.
 {code}
 $HBASE_HOME/bin/start-hbase.sh
 sleep 5 # bug HBASE-5666 rs doesn't retry if znode is not available.
 $HBASE_HOME/bin/local-regionservers.sh start 1 2 3
 {code}
 Once hbase is started I run an hbase shell script file (see below)
 everything is fine till last split operation.
 {code}
 # $HBASE_HOME/bin/hbase shell test.hbase
 # test.hbase
 create 'bugtb-t1', 'tcf11', 'tcf12'
 create 'bugtb-t2', 'tcf11', 'tcf12'
 put 'bugtb-t1', '10', 'tcf11:c1', 'a'
 put 'bugtb-t1', '15', 'tcf11:c2', 'b'
 put 'bugtb-t1', '20', 'tcf11:c1', 'c'
 put 'bugtb-t1', '30', 'tcf11:c2', 'd'
 put 'bugtb-t1', '35', 'tcf11:c1', 'e'
 put 'bugtb-t1', '40', 'tcf11:c2', 'f'
 put 'bugtb-t2', '10', 'tcf11:c1', 'a'
 put 'bugtb-t2', '15', 'tcf11:c2', 'b'
 put 'bugtb-t2', '20', 'tcf11:c1', 'c'
 put 'bugtb-t2', '30', 'tcf11:c2', 'd'
 put 'bugtb-t2', '35', 'tcf11:c1', 'e'
 put 'bugtb-t2', '40', 'tcf11:c2', 'f'
 split 'bugtb-t1', '20'
 split 'bugtb-t2', '20'
 split 'bugtb-t1', '40'
 {code}
 During the last split the region is still offline, and you get an 
 exception
 (If you sleep a bit before executing the last split, everything is fine)
 {code}
 ERROR: org.apache.hadoop.ipc.RemoteException: 
 org.apache.hadoop.hbase.NotServingRegionException: Region is not online: 
 bugtb-t1,,1333134892936.4e14c2cf4293156d5b099dc3d5c44890.
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3123)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2926)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:366)
   at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1383)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Resolved] (HBASE-5679) [89-fb] Port load test tool and related unit tests from trunk to 89-fb

2012-03-30 Thread Mikhail Bautin (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Bautin resolved HBASE-5679.
---

Resolution: Fixed

Committed internally.

 [89-fb] Port load test tool and related unit tests from trunk to 89-fb
 --

 Key: HBASE-5679
 URL: https://issues.apache.org/jira/browse/HBASE-5679
 Project: HBase
  Issue Type: Test
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin

 When open-sourcing LoadTestTool that originated in 89-fb, numerous 
 improvements to the tool were made, and unit tests based on the tool were 
 created as part of HBASE-4908. These improvements need to be ported back to 
 89-fb.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5619) Create PB protocols for HRegionInterface

2012-03-30 Thread Hadoop QA (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242764#comment-13242764
 ] 

Hadoop QA commented on HBASE-5619:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12520665/5619v6.txt
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 70 new Findbugs (version 
1.3.9) warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.client.TestFromClientSide
  org.apache.hadoop.hbase.mapreduce.TestImportTsv
  org.apache.hadoop.hbase.mapred.TestTableMapReduce
  org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1354//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1354//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1354//console

This message is automatically generated.

 Create PB protocols for HRegionInterface
 

 Key: HBASE-5619
 URL: https://issues.apache.org/jira/browse/HBASE-5619
 Project: HBase
  Issue Type: Sub-task
  Components: ipc, master, migration, regionserver
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.96.0

 Attachments: 5619v6.txt, 5619v6.txt, hbase-5619.patch, 
 hbase-5619_v3.patch, hbase-5619_v4.patch, hbase-5619_v5.patch


 Subtask of HBase-5443, separate HRegionInterface into admin protocol and 
 client protocol, create the PB protocol buffer files

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5084) Allow different HTable instances to share one ExecutorService

2012-03-30 Thread Lars Hofhansl (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242766#comment-13242766
 ] 

Lars Hofhansl commented on HBASE-5084:
--

Patch that allows passing just the ExecutorService.

 Allow different HTable instances to share one ExecutorService
 -

 Key: HBASE-5084
 URL: https://issues.apache.org/jira/browse/HBASE-5084
 Project: HBase
  Issue Type: Task
Reporter: Zhihong Yu
Assignee: Lars Hofhansl
 Fix For: 0.94.1

 Attachments: 5084-trunk.txt


 This came out of Lily 1.1.1 release:
 Use a shared ExecutorService for all HTable instances, leading to better (or 
 actual) thread reuse

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5084) Allow different HTable instances to share one ExecutorService

2012-03-30 Thread Lars Hofhansl (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5084:
-

Status: Patch Available  (was: Open)

 Allow different HTable instances to share one ExecutorService
 -

 Key: HBASE-5084
 URL: https://issues.apache.org/jira/browse/HBASE-5084
 Project: HBase
  Issue Type: Task
Reporter: Zhihong Yu
Assignee: Lars Hofhansl
 Fix For: 0.94.1

 Attachments: 5084-trunk.txt


 This came out of Lily 1.1.1 release:
 Use a shared ExecutorService for all HTable instances, leading to better (or 
 actual) thread reuse

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5084) Allow different HTable instances to share one ExecutorService

2012-03-30 Thread Lars Hofhansl (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5084:
-

Attachment: 5084-trunk.txt

 Allow different HTable instances to share one ExecutorService
 -

 Key: HBASE-5084
 URL: https://issues.apache.org/jira/browse/HBASE-5084
 Project: HBase
  Issue Type: Task
Reporter: Zhihong Yu
Assignee: Lars Hofhansl
 Fix For: 0.94.1

 Attachments: 5084-trunk.txt


 This came out of Lily 1.1.1 release:
 Use a shared ExecutorService for all HTable instances, leading to better (or 
 actual) thread reuse

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Reopened] (HBASE-5564) Bulkload is discarding duplicate records

2012-03-30 Thread Zhihong Yu (Reopened) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu reopened HBASE-5564:
---


By reverting the patch applied to trunk, 
TestImportTsv#testMROnTableWithCustomMapper passes.

 Bulkload is discarding duplicate records
 

 Key: HBASE-5564
 URL: https://issues.apache.org/jira/browse/HBASE-5564
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 0.96.0
 Environment: HBase 0.92
Reporter: Laxman
Assignee: Laxman
  Labels: bulkloader
 Fix For: 0.96.0

 Attachments: 5564.lint, 5564v5.txt, HBASE-5564_trunk.1.patch, 
 HBASE-5564_trunk.1.patch, HBASE-5564_trunk.2.patch, HBASE-5564_trunk.3.patch, 
 HBASE-5564_trunk.4_final.patch, HBASE-5564_trunk.patch


 Duplicate records are getting discarded when duplicate records exists in same 
 input file and more specifically if they exists in same split.
 Duplicate records are considered if the records are from diffrent different 
 splits.
 Version under test: HBase 0.92

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5619) Create PB protocols for HRegionInterface

2012-03-30 Thread stack (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5619:
-

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to trunk (hadoopqa doesn't seem to be picking this up... can fixup 
afterward if an issue).  Thanks for the patch Jimmy.

 Create PB protocols for HRegionInterface
 

 Key: HBASE-5619
 URL: https://issues.apache.org/jira/browse/HBASE-5619
 Project: HBase
  Issue Type: Sub-task
  Components: ipc, master, migration, regionserver
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.96.0

 Attachments: 5619v6.txt, 5619v6.txt, hbase-5619.patch, 
 hbase-5619_v3.patch, hbase-5619_v4.patch, hbase-5619_v5.patch


 Subtask of HBase-5443, separate HRegionInterface into admin protocol and 
 client protocol, create the PB protocol buffer files

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5564) Bulkload is discarding duplicate records

2012-03-30 Thread stack (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242785#comment-13242785
 ] 

stack commented on HBASE-5564:
--

Thanks Ted.  I reverted the patch for now.  Laxman, mind taking a looksee at 
the failures Ted found in TestImportTsv#testMROnTableWithCustomMapper?

 Bulkload is discarding duplicate records
 

 Key: HBASE-5564
 URL: https://issues.apache.org/jira/browse/HBASE-5564
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 0.96.0
 Environment: HBase 0.92
Reporter: Laxman
Assignee: Laxman
  Labels: bulkloader
 Fix For: 0.96.0

 Attachments: 5564.lint, 5564v5.txt, HBASE-5564_trunk.1.patch, 
 HBASE-5564_trunk.1.patch, HBASE-5564_trunk.2.patch, HBASE-5564_trunk.3.patch, 
 HBASE-5564_trunk.4_final.patch, HBASE-5564_trunk.patch


 Duplicate records are getting discarded when duplicate records exists in same 
 input file and more specifically if they exists in same split.
 Duplicate records are considered if the records are from diffrent different 
 splits.
 Version under test: HBase 0.92

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HBASE-5682) Add retry logic in HConnectionImplementation#resetZooKeeperTrackers (port to 0.94)

2012-03-30 Thread Lars Hofhansl (Created) (JIRA)
Add retry logic in HConnectionImplementation#resetZooKeeperTrackers (port to 
0.94)
--

 Key: HBASE-5682
 URL: https://issues.apache.org/jira/browse/HBASE-5682
 Project: HBase
  Issue Type: Sub-task
Reporter: Lars Hofhansl


Just realized that without this HBASE-4805 is broken.
I.e. there's no point keeping a persistent HConnection around if it can be 
rendered permanently unusable if the ZK connection is lost temporarily.
Note that this is fixed in 0.96 with HBASE-5399 (but that seems to big to 
backport)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HBASE-5683) Inconsistent behavior of bin/hbase scripts on different shell

2012-03-30 Thread Deepesh Khandelwal (Created) (JIRA)
Inconsistent behavior of bin/hbase scripts on different shell
-

 Key: HBASE-5683
 URL: https://issues.apache.org/jira/browse/HBASE-5683
 Project: HBase
  Issue Type: Bug
  Components: scripts
Affects Versions: 0.92.0
 Environment: GNU bash, version 3.2.51(1)-release 
(x86_64-suse-linux-gnu)
Reporter: Deepesh Khandelwal


When starting the hbase on the specific shell, observed that the HBase webapp 
at port 60010 was throwing a NOT FOUND error. The HBase server log was showing 
the following error:
2012-03-30 19:29:32,396 WARN org.mortbay.log: failed 
ContextHandlerCollection@67e2c841: java.lang.NoSuchFieldError: 
IS_SECURITY_ENABLED
2012-03-30 19:29:32,396 ERROR org.mortbay.log: Error starting handlers
java.lang.NoSuchFieldError: IS_SECURITY_ENABLED
   at 
org.apache.jasper.compiler.JspRuntimeContext.init(JspRuntimeContext.java:197)
   at org.apache.jasper.servlet.JspServlet.init(JspServlet.java:150)
   at 
org.mortbay.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:440)
   at 
org.mortbay.jetty.servlet.ServletHolder.doStart(ServletHolder.java:263)
   at 
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
   at 
org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:736)
   at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
   at 
org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
   at 
org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518)
   at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
   at 
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
   at 
org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
   at 
org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
   at 
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
   at 
org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
   at org.mortbay.jetty.Server.doStart(Server.java:224)
   at 
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
   at org.apache.hadoop.http.HttpServer.start(HttpServer.java:617)
   at 
org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:758)
   at 
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:479)
   at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:334)
   at java.lang.Thread.run(Thread.java:662)

The problem is apparently to do with the current classpath where we are using * 
to add multiple jars on the classpath. After following the same convention as 
bin/hadoop to add the individual jars, the errors are gone and the HBase webapp 
loads correctly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5619) Create PB protocols for HRegionInterface

2012-03-30 Thread stack (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242794#comment-13242794
 ] 

stack commented on HBASE-5619:
--

I tried TestFromClientSide a few times locally and its passing fine.

 Create PB protocols for HRegionInterface
 

 Key: HBASE-5619
 URL: https://issues.apache.org/jira/browse/HBASE-5619
 Project: HBase
  Issue Type: Sub-task
  Components: ipc, master, migration, regionserver
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.96.0

 Attachments: 5619v6.txt, 5619v6.txt, hbase-5619.patch, 
 hbase-5619_v3.patch, hbase-5619_v4.patch, hbase-5619_v5.patch


 Subtask of HBase-5443, separate HRegionInterface into admin protocol and 
 client protocol, create the PB protocol buffer files

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5666) RegionServer doesn't retry to check if base node is available

2012-03-30 Thread nkeywal (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242796#comment-13242796
 ] 

nkeywal commented on HBASE-5666:


I confirm I didn't modify this part in trunk. But who knows. I will have a look 
at it this week end.

 RegionServer doesn't retry to check if base node is available
 -

 Key: HBASE-5666
 URL: https://issues.apache.org/jira/browse/HBASE-5666
 Project: HBase
  Issue Type: Bug
  Components: regionserver, zookeeper
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Attachments: hbase-1-regionserver.log, hbase-2-regionserver.log, 
 hbase-3-regionserver.log, hbase-master.log, hbase-regionserver.log, 
 hbase-zookeeper.log


 I've a script that starts hbase and a couple of region servers in distributed 
 mode (hbase.cluster.distributed = true)
 {code}
 $HBASE_HOME/bin/start-hbase.sh
 $HBASE_HOME/bin/local-regionservers.sh start 1 2 3
 {code}
 but the region servers are not able to start...
 It seems that during the RS start the the znode is still not available, and 
 HRegionServer.initializeZooKeeper() check just once if the base not is 
 available.
 {code}
 2012-03-28 21:54:05,013 INFO 
 org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Check the value 
 configured in 'zookeeper.znode.parent'. There could be a mismatch with the 
 one configured in the master.
 2012-03-28 21:54:08,598 FATAL 
 org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server 
 localhost,60202,133296824: Initialization of RS failed.  Hence aborting 
 RS.
 java.io.IOException: Received the shutdown message while waiting.
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.blockAndCheckIfStopped(HRegionServer.java:626)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:596)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:558)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:672)
   at java.lang.Thread.run(Thread.java:662)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5666) RegionServer doesn't retry to check if base node is available

2012-03-30 Thread Matteo Bertozzi (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242801#comment-13242801
 ] 

Matteo Bertozzi commented on HBASE-5666:


yep, just checked and the code seems unchanged from trunk to 0.92...
Anyone has a different idea on that?
or I can try to implement a retry loop?

 RegionServer doesn't retry to check if base node is available
 -

 Key: HBASE-5666
 URL: https://issues.apache.org/jira/browse/HBASE-5666
 Project: HBase
  Issue Type: Bug
  Components: regionserver, zookeeper
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Attachments: hbase-1-regionserver.log, hbase-2-regionserver.log, 
 hbase-3-regionserver.log, hbase-master.log, hbase-regionserver.log, 
 hbase-zookeeper.log


 I've a script that starts hbase and a couple of region servers in distributed 
 mode (hbase.cluster.distributed = true)
 {code}
 $HBASE_HOME/bin/start-hbase.sh
 $HBASE_HOME/bin/local-regionservers.sh start 1 2 3
 {code}
 but the region servers are not able to start...
 It seems that during the RS start the the znode is still not available, and 
 HRegionServer.initializeZooKeeper() check just once if the base not is 
 available.
 {code}
 2012-03-28 21:54:05,013 INFO 
 org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Check the value 
 configured in 'zookeeper.znode.parent'. There could be a mismatch with the 
 one configured in the master.
 2012-03-28 21:54:08,598 FATAL 
 org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server 
 localhost,60202,133296824: Initialization of RS failed.  Hence aborting 
 RS.
 java.io.IOException: Received the shutdown message while waiting.
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.blockAndCheckIfStopped(HRegionServer.java:626)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:596)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:558)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:672)
   at java.lang.Thread.run(Thread.java:662)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HBASE-5684) Make ProcessBasedLocalHBaseCluster run HDFS and make it more robust

2012-03-30 Thread Mikhail Bautin (Created) (JIRA)
Make ProcessBasedLocalHBaseCluster run HDFS and make it more robust
---

 Key: HBASE-5684
 URL: https://issues.apache.org/jira/browse/HBASE-5684
 Project: HBase
  Issue Type: Improvement
Reporter: Mikhail Bautin
Assignee: Mikhail Bautin


Currently ProcessBasedLocalHBaseCluster runs on top of raw local filesystem. We 
need it to start a process-based HDFS cluster as well. We also need to make the 
whole thing more stable so we can use it in unit tests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




  1   2   >