date:20120203


[ 
https://issues.apache.org/jira/browse/HBASE-5328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199748#comment-13199748
 ] 

Zhihong Yu commented on HBASE-5328:
---

I think MockRegionServer.java should be placed under 
src/test/java/org/apache/hadoop/hbase/regionserver/

 Small changes to Master to make it more testable
 

 Key: HBASE-5328
 URL: https://issues.apache.org/jira/browse/HBASE-5328
 Project: HBase
  Issue Type: Task
Reporter: stack
 Attachments: 5328.txt


 Here are some small changes in Master that make it more testable.  Included 
 tests stand up a Master and then fake it into thinking that three 
 regionservers are registering making master assign root and meta, etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5329) when client call lockRow,region server may allocate duplicated lock id, this may cause that client is blocked.


 [ 
https://issues.apache.org/jira/browse/HBASE-5329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5329:
--

Description: 
{code}
protected long addRowLock(Integer r, HRegion region) throws 
LeaseStillHeldException
{
long lockId = -1L;
lockId = rand.nextLong();   //!!!may generate duplicated 
id，bug?
String lockName = String.valueOf(lockId);
rowlocks.put(lockName, r);
this.leases.createLease(lockName, new RowLockListener(lockName, 
region));
return lockId;
}
{code}

In addRowLock(),rand may generate duplicated lock id, it may cause regionserver 
throw exception(Leases$LeaseStillHeldException).The client will be blocked 
until old rowlock is released.

{code}
2012-02-03 15:21:50,084 ERROR 
org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock 
(fsOk: true)
org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException
at 
org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:1986)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:1963)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
at 
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
{code}

  was:
protected long addRowLock(Integer r, HRegion region) throws 
LeaseStillHeldException
{
long lockId = -1L;
lockId = rand.nextLong();   //!!!may generate duplicated 
id，bug?
String lockName = String.valueOf(lockId);
rowlocks.put(lockName, r);
this.leases.createLease(lockName, new RowLockListener(lockName, 
region));
return lockId;
}

In addRowLock(),rand may generate duplicated lock id, it may cause regionserver 
throw exception(Leases$LeaseStillHeldException).The client will be blocked 
until old rowlock is released.


2012-02-03 15:21:50,084 ERROR 
org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock 
(fsOk: true)
org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException
at 
org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:1986)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:1963)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
at 
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)



I think we should call putIfAbsent().
If the return value from putIfAbsent() is not null, we should try to regenerate 
a new lockId.

Do you want to upload a patch ?

 when client call lockRow,region server may allocate duplicated lock id, this 
 may cause that client  is blocked.
 ---

 Key: HBASE-5329
 URL: https://issues.apache.org/jira/browse/HBASE-5329
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.3
 Environment: Red Hat Enterprise Linux Server release 5.4 
Reporter: liaoxiangui

 {code}
 protected long addRowLock(Integer r, HRegion region) throws 
 LeaseStillHeldException
 {
   long lockId = -1L;
   lockId = rand.nextLong();   //!!!may generate duplicated 
 id，bug?
   String lockName = String.valueOf(lockId);
   rowlocks.put(lockName, r);
   this.leases.createLease(lockName, new RowLockListener(lockName, 
 region));
   return lockId;
 }
 {code}
 In addRowLock(),rand may generate duplicated lock id, it may cause 
 regionserver throw exception(Leases$LeaseStillHeldException).The client will 
 be blocked until old rowlock is released.
 {code}
 2012-02-03 15:21:50,084 ERROR 
 org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock 
 (fsOk: true)
 org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException
 at 
 org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150)
 at

[jira] [Updated] (HBASE-5329) when client call lockRow,region server may allocate duplicated lock id, this may cause that client is blocked.


 [ 
https://issues.apache.org/jira/browse/HBASE-5329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5329:
--

Priority: Critical  (was: Major)

 when client call lockRow,region server may allocate duplicated lock id, this 
 may cause that client  is blocked.
 ---

 Key: HBASE-5329
 URL: https://issues.apache.org/jira/browse/HBASE-5329
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.3
 Environment: Red Hat Enterprise Linux Server release 5.4 
Reporter: liaoxiangui
Priority: Critical

 {code}
 protected long addRowLock(Integer r, HRegion region) throws 
 LeaseStillHeldException
 {
   long lockId = -1L;
   lockId = rand.nextLong();   //!!!may generate duplicated 
 id，bug?
   String lockName = String.valueOf(lockId);
   rowlocks.put(lockName, r);
   this.leases.createLease(lockName, new RowLockListener(lockName, 
 region));
   return lockId;
 }
 {code}
 In addRowLock(),rand may generate duplicated lock id, it may cause 
 regionserver throw exception(Leases$LeaseStillHeldException).The client will 
 be blocked until old rowlock is released.
 {code}
 2012-02-03 15:21:50,084 ERROR 
 org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock 
 (fsOk: true)
 org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException
 at 
 org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:1986)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:1963)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5329) addRowLock() may allocate duplicate lock id, causing the client to be blocked

2012-02-03 Thread ramkrishna.s.vasudevan (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5329:
--

Summary: addRowLock() may allocate duplicate lock id, causing the client to 
be blocked  (was: when client call lockRow,region server may allocate 
duplicated lock id, this may cause that client  is blocked.)

 addRowLock() may allocate duplicate lock id, causing the client to be blocked
 -

 Key: HBASE-5329
 URL: https://issues.apache.org/jira/browse/HBASE-5329
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.3
 Environment: Red Hat Enterprise Linux Server release 5.4 
Reporter: liaoxiangui
Priority: Critical

 {code}
 protected long addRowLock(Integer r, HRegion region) throws 
 LeaseStillHeldException
 {
   long lockId = -1L;
   lockId = rand.nextLong();   //!!!may generate duplicated 
 id，bug?
   String lockName = String.valueOf(lockId);
   rowlocks.put(lockName, r);
   this.leases.createLease(lockName, new RowLockListener(lockName, 
 region));
   return lockId;
 }
 {code}
 In addRowLock(),rand may generate duplicated lock id, it may cause 
 regionserver throw exception(Leases$LeaseStillHeldException).The client will 
 be blocked until old rowlock is released.
 {code}
 2012-02-03 15:21:50,084 ERROR 
 org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock 
 (fsOk: true)
 org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException
 at 
 org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:1986)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:1963)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5328) Small changes to Master to make it more testable

2012-02-03 Thread stack (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199888#comment-13199888
 ] 

stack commented on HBASE-5328:
--

bq. I think MockRegionServer.java should be placed under 
src/test/java/org/apache/hadoop/hbase/regionserver/

Not yet.  Its built for testing the Master (as it says on the class comment).  
Later when it gets flushed out more it might make sense being moved elsewhere.

Thanks for review.  Hold off though till its more finished.  Good on you Ted.



 Small changes to Master to make it more testable
 

 Key: HBASE-5328
 URL: https://issues.apache.org/jira/browse/HBASE-5328
 Project: HBase
  Issue Type: Task
Reporter: stack
 Attachments: 5328.txt


 Here are some small changes in Master that make it more testable.  Included 
 tests stand up a Master and then fake it into thinking that three 
 regionservers are registering making master assign root and meta, etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5323) Need to handle assertion error while splitting log through ServerShutDownHandler by shutting down the master


 [ 
https://issues.apache.org/jira/browse/HBASE-5323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5323:
--

Attachment: HBASE-5323.patch

Patch for 0.90.

 Need to handle assertion error while splitting log through 
 ServerShutDownHandler by shutting down the master
 

 Key: HBASE-5323
 URL: https://issues.apache.org/jira/browse/HBASE-5323
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.5
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.94.0, 0.90.7

 Attachments: HBASE-5323.patch


 We know that while parsing the HLog we expect the proper length from HDFS.
 In WALReaderFSDataInputStream
 {code}
   assert(realLength = this.length);
 {code}
 We are trying to come out if the above condition is not satisfied.  But if 
 SSH.splitLog() gets this problem then it lands in the run method of 
 EventHandler.  This kills the SSH thread and so further assignment does not 
 happen.  If ROOT and META are to be assigned they cannot be.
 I think in this condition we abort the master by catching such exceptions.
 Please do suggest.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5200) AM.ProcessRegionInTransition() and AM.handleRegion() race thus leaving the region assignment inconsistent


 [ 
https://issues.apache.org/jira/browse/HBASE-5200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5200:
--

Attachment: 5200-v2.txt

Patch v2 incorporates review comments

 AM.ProcessRegionInTransition() and AM.handleRegion() race thus leaving the 
 region assignment inconsistent
 -

 Key: HBASE-5200
 URL: https://issues.apache.org/jira/browse/HBASE-5200
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.5
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.94.0, 0.90.7, 0.92.1

 Attachments: 5200-v2.txt, HBASE-5200.patch, HBASE-5200_1.patch, 
 TEST-org.apache.hadoop.hbase.master.TestRestartCluster.xml


 This is the scenario
 Consider a case where the balancer is going on thus trying to close regions 
 in a RS.
 Before we could close a master switch happens.  
 On Master switch the set of nodes that are in RIT is collected and we first 
 get Data and start watching the node
 After that the node data is added into RIT.
 Now by this time (before adding to RIT) if the RS to which close was called 
 does a transition in AM.handleRegion() we miss the handling saying RIT state 
 was null.
 {code}
 2012-01-13 10:50:46,358 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 a66d281d231dfcaea97c270698b26b6f from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,358 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 c12e53bfd48ddc5eec507d66821c4d23 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,358 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 59ae13de8c1eb325a0dd51f4902d2052 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 f45bc9614d7575f35244849af85aa078 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 cc3ecd7054fe6cd4a1159ed92fd62641 from server 
 HOST-192-168-47-204,20020,1326342744518 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 3af40478a17fee96b4a192b22c90d5a2 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 e6096a8466e730463e10d3d61f809b92 from server 
 HOST-192-168-47-204,20020,1326342744518 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 4806781a1a23066f7baed22b4d237e24 from server 
 HOST-192-168-47-204,20020,1326342744518 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 d69e104131accaefe21dcc01fddc7629 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 {code}
 In branch the CLOSING node is created by RS thus leading to more 
 inconsistency.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5323) Need to handle assertion error while splitting log through ServerShutDownHandler by shutting down the master

2012-02-03 Thread Todd Lipcon (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199917#comment-13199917
]

Todd Lipcon commented on HBASE-5323:

If this is an actual error condition that can be seen in practice (rather than
indication of a failed programmer assumption) it should be changed to an
if/throw, not an assert. Most users don't have asserts enabled in production.

Need to handle assertion error while splitting log through
ServerShutDownHandler by shutting down the master

Key: HBASE-5323
URL: https://issues.apache.org/jira/browse/HBASE-5323
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.5
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.94.0, 0.90.7

Attachments: HBASE-5323.patch

We know that while parsing the HLog we expect the proper length from HDFS.
In WALReaderFSDataInputStream
{code}
assert(realLength = this.length);
{code}
We are trying to come out if the above condition is not satisfied. But if
SSH.splitLog() gets this problem then it lands in the run method of
EventHandler. This kills the SSH thread and so further assignment does not
happen. If ROOT and META are to be assigned they cannot be.
I think in this condition we abort the master by catching such exceptions.
Please do suggest.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5318) Support Eclipse Indigo

2012-02-03 Thread Kannan Muthukkaruppan (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199924#comment-13199924
]

Jesse Yates commented on HBASE-5318:

@stack: I'll look into seeing if something useful can be put in. Patch coming
this afternoon.

Support Eclipse Indigo
---

Key: HBASE-5318
URL: https://issues.apache.org/jira/browse/HBASE-5318
Project: HBase
Issue Type: Improvement
Components: build
Affects Versions: 0.94.0
Environment: Eclipse Indigo (1.4.1) which includes m2eclipse (1.0
SR1).
Reporter: Jesse Yates
Assignee: Jesse Yates
Priority: Minor
Labels: maven
Attachments: mvn_HBASE-5318_r0.patch

The current 'standard' release of Eclipse (indigo) comes with m2eclipse
installed. However, as of m2e v1.0, interesting lifecycle phases are now
handled via a 'connector'. However, several of the plugins we use don't
support connectors. This means that eclipse bails out and won't build the
project or view it as 'working' even though it builds just fine from the the
command line.
Since Eclipse is one of the major java IDEs and that Indigo has been around
for a while, we should make it easy to for new devs to pick up the code and
for older devs to upgrade painlessly. The original build should not be
modified in any significant way.

[jira] [Commented] (HBASE-4608) HLog Compression


[ 
https://issues.apache.org/jira/browse/HBASE-4608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199925#comment-13199925
 ] 

Kannan Muthukkaruppan commented on HBASE-4608:
--

Li: Is there a writeup/description of the scheme that this patch is 
implementing? If not, would you mind giving a quick overview. Thanks much.

 HLog Compression
 

 Key: HBASE-4608
 URL: https://issues.apache.org/jira/browse/HBASE-4608
 Project: HBase
  Issue Type: New Feature
Reporter: Li Pi
Assignee: Li Pi
 Attachments: 4608v1.txt, 4608v5.txt, 4608v6.txt, 4608v7.txt, 
 4608v8fixed.txt


 The current bottleneck to HBase write speed is replicating the WAL appends 
 across different datanodes. We can speed up this process by compressing the 
 HLog. Current plan involves using a dictionary to compress table name, region 
 id, cf name, and possibly other bits of repeated data. Also, HLog format may 
 be changed in other ways to produce a smaller HLog.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4658) Put attributes are not exposed via the ThriftServer

2012-02-03 Thread dhruba borthakur (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199926#comment-13199926
 ] 

dhruba borthakur commented on HBASE-4658:
-

I think the unit test failures are not related to my patch. Does this mean that 
this patch is good to go?

 Put attributes are not exposed via the ThriftServer
 ---

 Key: HBASE-4658
 URL: https://issues.apache.org/jira/browse/HBASE-4658
 Project: HBase
  Issue Type: Bug
  Components: thrift
Reporter: dhruba borthakur
Assignee: dhruba borthakur
 Attachments: D1563.1.patch, D1563.1.patch, D1563.1.patch, 
 D1563.2.patch, D1563.2.patch, D1563.2.patch, D1563.3.patch, D1563.3.patch, 
 D1563.3.patch, ThriftPutAttributes1.txt


 The Put api also takes in a bunch of arbitrary attributes that an application 
 can use to associate metadata with each put operation. This is not exposed 
 via Thrift.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5229) Explore building blocks for multi-row local transactions.

2012-02-03 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199933#comment-13199933
 ] 

jirapos...@reviews.apache.org commented on HBASE-5229:
--



bq.  On 2012-02-03 05:07:24, Ted Yu wrote:
bq.   
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java,
 line 3160
bq.   https://reviews.apache.org/r/3748/diff/1/?file=72045#file72045line3160
bq.  
bq.   My point was we shouldn't throw exception in this case.
bq.   MultiRowMutation should be delivered to the correct region.
bq.   
bq.   I think we agree on the above.

Oh... MultiRowMutation does not know anything about regions. So we have the use 
the one passed.


- Lars


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3748/#review4791
---


On 2012-02-03 01:29:58, Lars Hofhansl wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/3748/
bq.  ---
bq.  
bq.  (Updated 2012-02-03 01:29:58)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This builds on HBASE-3584, HBASE-5203, and HBASE-5304.
bq.  
bq.  Multiple Rows can be locked and applied atomically as long as the 
application ensures that all rows reside in the same Region (by presplitting or 
a custom RegionSplitPolicy).
bq.  At SFDC we can use this to colocate subsets of a tenant's data and allow 
atomic operations over these subsets.
bq.  
bq.  Obviously this is an advanced features and this prominently called out in 
the Javadoc.
bq.  
bq.  
bq.  This addresses bug HBASE-5229.
bq.  https://issues.apache.org/jira/browse/HBASE-5229
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiRowMutation.java
 PRE-CREATION 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java
 1239953 
bq.  
bq.  Diff: https://reviews.apache.org/r/3748/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  Tests added to TestFromClientSide and TestAtomicOperation
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Lars
bq.  
bq.



 Explore building blocks for multi-row local transactions.
 ---

 Key: HBASE-5229
 URL: https://issues.apache.org/jira/browse/HBASE-5229
 Project: HBase
  Issue Type: New Feature
  Components: client, regionserver
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.0

 Attachments: 5229-multiRow.txt, 5229-seekto-v2.txt, 5229-seekto.txt, 
 5229.txt


 HBase should provide basic building blocks for multi-row local transactions. 
 Local means that we do this by co-locating the data. Global (cross region) 
 transactions are not discussed here.
 After a bit of discussion two solutions have emerged:
 1. Keep the row-key for determining grouping and location and allow efficient 
 intra-row scanning. A client application would then model tables as 
 HBase-rows.
 2. Define a prefix-length in HTableDescriptor that defines a grouping of 
 rows. Regions will then never be split inside a grouping prefix.
 #1 is true to the current storage paradigm

[jira] [Commented] (HBASE-5229) Explore building blocks for multi-row local transactions.

2012-02-03 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199932#comment-13199932
 ] 

jirapos...@reviews.apache.org commented on HBASE-5229:
--



bq.  On 2012-02-03 05:01:34, Jimmy Xiang wrote:
bq.   
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiRowMutation.java,
 line 67
bq.   https://reviews.apache.org/r/3748/diff/1/?file=72039#file72039line67
bq.  
bq.   Should we check they are for different rows here?

It is still valid to pass the same rows here.


bq.  On 2012-02-03 05:01:34, Jimmy Xiang wrote:
bq.   
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java,
 line 46
bq.   https://reviews.apache.org/r/3748/diff/1/?file=72040#file72040line46
bq.  
bq.   This is called by default.  No need to call it explicitly.

rights... will fix.


- Lars


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3748/#review4789
---


On 2012-02-03 01:29:58, Lars Hofhansl wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/3748/
bq.  ---
bq.  
bq.  (Updated 2012-02-03 01:29:58)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This builds on HBASE-3584, HBASE-5203, and HBASE-5304.
bq.  
bq.  Multiple Rows can be locked and applied atomically as long as the 
application ensures that all rows reside in the same Region (by presplitting or 
a custom RegionSplitPolicy).
bq.  At SFDC we can use this to colocate subsets of a tenant's data and allow 
atomic operations over these subsets.
bq.  
bq.  Obviously this is an advanced features and this prominently called out in 
the Javadoc.
bq.  
bq.  
bq.  This addresses bug HBASE-5229.
bq.  https://issues.apache.org/jira/browse/HBASE-5229
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiRowMutation.java
 PRE-CREATION 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java
 1239953 
bq.  
bq.  Diff: https://reviews.apache.org/r/3748/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  Tests added to TestFromClientSide and TestAtomicOperation
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Lars
bq.  
bq.



 Explore building blocks for multi-row local transactions.
 ---

 Key: HBASE-5229
 URL: https://issues.apache.org/jira/browse/HBASE-5229
 Project: HBase
  Issue Type: New Feature
  Components: client, regionserver
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.0

 Attachments: 5229-multiRow.txt, 5229-seekto-v2.txt, 5229-seekto.txt, 
 5229.txt


 HBase should provide basic building blocks for multi-row local transactions. 
 Local means that we do this by co-locating the data. Global (cross region) 
 transactions are not discussed here.
 After a bit of discussion two solutions have emerged:
 1. Keep the row-key for determining grouping and location and allow efficient 
 intra-row scanning. A client application would then model tables as 
 HBase-rows.
 2. Define a

[jira] [Commented] (HBASE-5229) Explore building blocks for multi-row local transactions.

2012-02-03 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199934#comment-13199934
 ] 

jirapos...@reviews.apache.org commented on HBASE-5229:
--



bq.  On 2012-02-03 04:45:54, Ted Yu wrote:
bq.   
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java,
 line 39
bq.   https://reviews.apache.org/r/3748/diff/1/?file=72040#file72040line39
bq.  
bq.   From its name, RowMutation seems to refer to single row. It is a 
little confusing RowMutation extends MultiRowMutation.
bq.  
bq.  Lars Hofhansl wrote:
bq.  Yeah. Maybe I'll have a common super class instead.

On 2nd thought. RowMutation *is* specialisation of MultiRowMutation, one that 
only allows a single row.


- Lars


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3748/#review4788
---


On 2012-02-03 01:29:58, Lars Hofhansl wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/3748/
bq.  ---
bq.  
bq.  (Updated 2012-02-03 01:29:58)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This builds on HBASE-3584, HBASE-5203, and HBASE-5304.
bq.  
bq.  Multiple Rows can be locked and applied atomically as long as the 
application ensures that all rows reside in the same Region (by presplitting or 
a custom RegionSplitPolicy).
bq.  At SFDC we can use this to colocate subsets of a tenant's data and allow 
atomic operations over these subsets.
bq.  
bq.  Obviously this is an advanced features and this prominently called out in 
the Javadoc.
bq.  
bq.  
bq.  This addresses bug HBASE-5229.
bq.  https://issues.apache.org/jira/browse/HBASE-5229
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiRowMutation.java
 PRE-CREATION 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java
 1239953 
bq.  
bq.  Diff: https://reviews.apache.org/r/3748/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  Tests added to TestFromClientSide and TestAtomicOperation
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Lars
bq.  
bq.



 Explore building blocks for multi-row local transactions.
 ---

 Key: HBASE-5229
 URL: https://issues.apache.org/jira/browse/HBASE-5229
 Project: HBase
  Issue Type: New Feature
  Components: client, regionserver
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.0

 Attachments: 5229-multiRow.txt, 5229-seekto-v2.txt, 5229-seekto.txt, 
 5229.txt


 HBase should provide basic building blocks for multi-row local transactions. 
 Local means that we do this by co-locating the data. Global (cross region) 
 transactions are not discussed here.
 After a bit of discussion two solutions have emerged:
 1. Keep the row-key for determining grouping and location and allow efficient 
 intra-row scanning. A client application would then model tables as 
 HBase-rows.
 2. Define a prefix-length in HTableDescriptor that defines a grouping of 
 rows. Regions will then never be split inside a grouping prefix.
 #1 is true

[jira] [Commented] (HBASE-5229) Explore building blocks for multi-row local transactions.

2012-02-03 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199940#comment-13199940
 ] 

jirapos...@reviews.apache.org commented on HBASE-5229:
--



bq.  On 2012-02-03 04:45:54, Ted Yu wrote:
bq.   
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java,
 line 96
bq.   https://reviews.apache.org/r/3748/diff/1/?file=72040#file72040line96
bq.  
bq.   version would be read / written twice, right ?
bq.  
bq.  Lars Hofhansl wrote:
bq.  Yes. Need to fix that.

And actually this is what I had in mind too. Both MultiRowMutation and 
RowMution write their version.
MultiRowMutation knowns to serialize mutations, that process needs to be 
versioned.
RowMutation adds some fields on top of that, those also need to be versioned.

Maybe the two should be entirely independent (at the expense of a few 
duplicated lines of code) so that they can evolve independently.


- Lars


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3748/#review4788
---


On 2012-02-03 01:29:58, Lars Hofhansl wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/3748/
bq.  ---
bq.  
bq.  (Updated 2012-02-03 01:29:58)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This builds on HBASE-3584, HBASE-5203, and HBASE-5304.
bq.  
bq.  Multiple Rows can be locked and applied atomically as long as the 
application ensures that all rows reside in the same Region (by presplitting or 
a custom RegionSplitPolicy).
bq.  At SFDC we can use this to colocate subsets of a tenant's data and allow 
atomic operations over these subsets.
bq.  
bq.  Obviously this is an advanced features and this prominently called out in 
the Javadoc.
bq.  
bq.  
bq.  This addresses bug HBASE-5229.
bq.  https://issues.apache.org/jira/browse/HBASE-5229
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiRowMutation.java
 PRE-CREATION 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java
 1239953 
bq.  
bq.  Diff: https://reviews.apache.org/r/3748/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  Tests added to TestFromClientSide and TestAtomicOperation
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Lars
bq.  
bq.



 Explore building blocks for multi-row local transactions.
 ---

 Key: HBASE-5229
 URL: https://issues.apache.org/jira/browse/HBASE-5229
 Project: HBase
  Issue Type: New Feature
  Components: client, regionserver
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.0

 Attachments: 5229-multiRow.txt, 5229-seekto-v2.txt, 5229-seekto.txt, 
 5229.txt


 HBase should provide basic building blocks for multi-row local transactions. 
 Local means that we do this by co-locating the data. Global (cross region) 
 transactions are not discussed here.
 After a bit of discussion two solutions have emerged:
 1. Keep the row-key for determining grouping and location and allow efficient 
 intra-row scanning. A client application

[jira] [Commented] (HBASE-4336) Convert source tree into maven modules


[ 
https://issues.apache.org/jira/browse/HBASE-4336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199941#comment-13199941
 ] 

Jesse Yates commented on HBASE-4336:


@stack
{quote}
For this kind of work I'd suggest something like restructure101. They are a 
good crew and have given apache hbase a license; I could ask them for one for 
you.
{quote}

I've tried a couple of free options out there, but nothing was able to really 
scale to handle the hbase code.  I'd love to try using restructure101 to dig 
through this.

{quote}
and then the Util class would be hoisted up into the packages above so it could 
be used from multiple packages
{quote}

The utils would be hoisted into the 'shared' module, so we can avoid having 
dependencies between things like the client and server modules. This means that 
shared gets heavily cross-referenced, but that complexity is bound to a single 
location, rather than being tied all over the place (like it is now, smile).

{quote}
I'm not a fan of building up HConstants to be this fat Interface refereneced by 
all and sundry
{quote}

My logic here is the same as the previous - trying to reduce the 
interpackage/module complexity by hoisting out common elements. I agree that 
things that are just used _within_ a class should remain with that class (as 
private members), but when you start getting things across packages making 
references to this file, it gets *really* hard to keep track of which constants 
you should be using where. For instance, the craziness I found with the 
DEFAULT_BLOCK_SIZE, even though that has apparently been removed, its 
indicative of the bigger problem. 

Instead, for things like this, it is really nice to have a single place to go 
to find the right constant. Doing something like 
HConstants.HFile.DEFAULT_BLOCK_SIZE actually feels really clean when done from 
within an IDE. HConstants is already a pretty massive file, and the efforts to 
keep constants in their respective classes has only lead to more pain (And 
probably some duplication), rather than making it easier and cleaner. 

{quote}
An org.apache.hadoop.hbase.constants
{quote}

Yeah, that was the idea. It would help break the constants into a couple 
different files, but at the same time keep them more together. Problem here is 
that you will likely end up having do an inline full path specification of the 
class to differentiate it from the actual class, where you are probably using 
it, (eg. org.apache.hadoop.hbase.constants.HFile.DEFAULT_BLOCK_SIZE), which is 
annoying.   

 Convert source tree into maven modules
 --

 Key: HBASE-4336
 URL: https://issues.apache.org/jira/browse/HBASE-4336
 Project: HBase
  Issue Type: Task
  Components: build
Reporter: Gary Helmling
Priority: Critical
 Fix For: 0.94.0


 When we originally converted the build to maven we had a single core module 
 defined, but later reverted this to a module-less build for the sake of 
 simplicity.
 It now looks like it's time to re-address this, as we have an actual need for 
 modules to:
 * provide a trimmed down client library that applications can make use of
 * more cleanly support building against different versions of Hadoop, in 
 place of some of the reflection machinations currently required
 * incorporate the secure RPC engine that depends on some secure Hadoop classes
 I propose we start simply by refactoring into two initial modules:
 * core - common classes and utilities, and client-side code and interfaces
 * server - master and region server implementations and supporting code
 This would also lay the groundwork for incorporating the HBase security 
 features that have been developed.  Once the module structure is in place, 
 security-related features could then be incorporated into a third module -- 
 security -- after normal review and approval.  The security module could 
 then depend on secure Hadoop, without modifying the dependencies of the rest 
 of the HBase code.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5330) TestCompactSelection - adding 2 test cases to testCompactionRatio

2012-02-03 Thread Doug Meil (Created) (JIRA)

TestCompactSelection - adding 2 test cases to testCompactionRatio
-

 Key: HBASE-5330
 URL: https://issues.apache.org/jira/browse/HBASE-5330
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor


There were three existing assertions in the compaction test that did max # of 
files assertions...
{code}
assertEquals(maxFiles,

store.compactSelection(sfCreate(7,6,5,4,3,2,1)).getFilesToCompact().size());
{code}

... and for references ...

{code}
  assertEquals(maxFiles,
store.compactSelection(sfCreate(true, 
7,6,5,4,3,2,1)).getFilesToCompact().size());
{code}
 
... but they didn't assert against which StoreFiles got selected.  While the 
number of StoreFiles is the same, the files selected are actually different, 
and I thought that there should be explicit assertions showing that.



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5330) TestCompactSelection - adding 2 test cases to testCompactionRatio

2012-02-03 Thread Doug Meil (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-5330:
-

Description: 
There were three existing assertions in TestCompactSelection 
testCompactionRatio that did max # of files assertions...
{code}
assertEquals(maxFiles,

store.compactSelection(sfCreate(7,6,5,4,3,2,1)).getFilesToCompact().size());
{code}

... and for references ...

{code}
  assertEquals(maxFiles,
store.compactSelection(sfCreate(true, 
7,6,5,4,3,2,1)).getFilesToCompact().size());
{code}
 
... but they didn't assert against which StoreFiles got selected.  While the 
number of StoreFiles is the same, the files selected are actually different, 
and I thought that there should be explicit assertions showing that.



  was:
There were three existing assertions in the compaction test that did max # of 
files assertions...
{code}
assertEquals(maxFiles,

store.compactSelection(sfCreate(7,6,5,4,3,2,1)).getFilesToCompact().size());
{code}

... and for references ...

{code}
  assertEquals(maxFiles,
store.compactSelection(sfCreate(true, 
7,6,5,4,3,2,1)).getFilesToCompact().size());
{code}
 
... but they didn't assert against which StoreFiles got selected.  While the 
number of StoreFiles is the same, the files selected are actually different, 
and I thought that there should be explicit assertions showing that.




 TestCompactSelection - adding 2 test cases to testCompactionRatio
 -

 Key: HBASE-5330
 URL: https://issues.apache.org/jira/browse/HBASE-5330
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor

 There were three existing assertions in TestCompactSelection 
 testCompactionRatio that did max # of files assertions...
 {code}
 assertEquals(maxFiles,
 
 store.compactSelection(sfCreate(7,6,5,4,3,2,1)).getFilesToCompact().size());
 {code}
 ... and for references ...
 {code}
   assertEquals(maxFiles,
 store.compactSelection(sfCreate(true, 
 7,6,5,4,3,2,1)).getFilesToCompact().size());
 {code}
  
 ... but they didn't assert against which StoreFiles got selected.  While the 
 number of StoreFiles is the same, the files selected are actually different, 
 and I thought that there should be explicit assertions showing that.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5330) TestCompactSelection - adding 2 test cases to testCompactionRatio

2012-02-03 Thread Doug Meil (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-5330:
-

Attachment: TestCompactSelection_hbase_5330.java.patch

 TestCompactSelection - adding 2 test cases to testCompactionRatio
 -

 Key: HBASE-5330
 URL: https://issues.apache.org/jira/browse/HBASE-5330
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: TestCompactSelection_hbase_5330.java.patch


 There were three existing assertions in TestCompactSelection 
 testCompactionRatio that did max # of files assertions...
 {code}
 assertEquals(maxFiles,
 
 store.compactSelection(sfCreate(7,6,5,4,3,2,1)).getFilesToCompact().size());
 {code}
 ... and for references ...
 {code}
   assertEquals(maxFiles,
 store.compactSelection(sfCreate(true, 
 7,6,5,4,3,2,1)).getFilesToCompact().size());
 {code}
  
 ... but they didn't assert against which StoreFiles got selected.  While the 
 number of StoreFiles is the same, the files selected are actually different, 
 and I thought that there should be explicit assertions showing that.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5330) TestCompactSelection - adding 2 test cases to testCompactionRatio

2012-02-03 Thread Doug Meil (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doug Meil updated HBASE-5330:
-

Status: Patch Available  (was: Open)

 TestCompactSelection - adding 2 test cases to testCompactionRatio
 -

 Key: HBASE-5330
 URL: https://issues.apache.org/jira/browse/HBASE-5330
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: TestCompactSelection_hbase_5330.java.patch


 There were three existing assertions in TestCompactSelection 
 testCompactionRatio that did max # of files assertions...
 {code}
 assertEquals(maxFiles,
 
 store.compactSelection(sfCreate(7,6,5,4,3,2,1)).getFilesToCompact().size());
 {code}
 ... and for references ...
 {code}
   assertEquals(maxFiles,
 store.compactSelection(sfCreate(true, 
 7,6,5,4,3,2,1)).getFilesToCompact().size());
 {code}
  
 ... but they didn't assert against which StoreFiles got selected.  While the 
 number of StoreFiles is the same, the files selected are actually different, 
 and I thought that there should be explicit assertions showing that.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5229) Explore building blocks for multi-row local transactions.

[
https://issues.apache.org/jira/browse/HBASE-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199946#comment-13199946
]

Zhihong Yu commented on HBASE-5229:
---

bq. Maybe the two should be entirely independent (at the expense of a few
duplicated lines of code) so that they can evolve independently.
That is better.

Explore building blocks for multi-row local transactions.
---

Key: HBASE-5229
URL: https://issues.apache.org/jira/browse/HBASE-5229
Project: HBase
Issue Type: New Feature
Components: client, regionserver
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.94.0

Attachments: 5229-multiRow.txt, 5229-seekto-v2.txt, 5229-seekto.txt,
5229.txt

HBase should provide basic building blocks for multi-row local transactions.
Local means that we do this by co-locating the data. Global (cross region)
transactions are not discussed here.
After a bit of discussion two solutions have emerged:
1. Keep the row-key for determining grouping and location and allow efficient
intra-row scanning. A client application would then model tables as
HBase-rows.
2. Define a prefix-length in HTableDescriptor that defines a grouping of
rows. Regions will then never be split inside a grouping prefix.
#1 is true to the current storage paradigm of HBase.
#2 is true to the current client side API.
I will explore these two with sample patches here.

Was:
As discussed (at length) on the dev mailing list with the HBASE-3584 and
HBASE-5203 committed, supporting atomic cross row transactions within a
region becomes simple.
I am aware of the hesitation about the usefulness of this feature, but we
have to start somewhere.
Let's use this jira for discussion, I'll attach a patch (with tests)
momentarily to make this concrete.

[jira] [Commented] (HBASE-4658) Put attributes are not exposed via the ThriftServer


[ 
https://issues.apache.org/jira/browse/HBASE-4658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199948#comment-13199948
 ] 

Zhihong Yu commented on HBASE-4658:
---

Integrated to TRUNK.

Thanks for the patch, Dhruba.

Thanks for the review, Scott.

 Put attributes are not exposed via the ThriftServer
 ---

 Key: HBASE-4658
 URL: https://issues.apache.org/jira/browse/HBASE-4658
 Project: HBase
  Issue Type: Bug
  Components: thrift
Reporter: dhruba borthakur
Assignee: dhruba borthakur
 Attachments: D1563.1.patch, D1563.1.patch, D1563.1.patch, 
 D1563.2.patch, D1563.2.patch, D1563.2.patch, D1563.3.patch, D1563.3.patch, 
 D1563.3.patch, ThriftPutAttributes1.txt


 The Put api also takes in a bunch of arbitrary attributes that an application 
 can use to associate metadata with each put operation. This is not exposed 
 via Thrift.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5186) Add metrics to ThriftServer


 [ 
https://issues.apache.org/jira/browse/HBASE-5186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5186:
--

Resolution: Fixed
Status: Resolved  (was: Patch Available)

 Add metrics to ThriftServer
 ---

 Key: HBASE-5186
 URL: https://issues.apache.org/jira/browse/HBASE-5186
 Project: HBase
  Issue Type: Improvement
Reporter: Scott Chen
Assignee: Scott Chen
 Fix For: 0.94.0

 Attachments: 5186-v10.txt, 5186-v11.txt, 5186-v12.txt, 5186-v9.txt, 
 HBASE-5186.D1461.1.patch, HBASE-5186.D1461.2.patch, HBASE-5186.D1461.3.patch, 
 HBASE-5186.D1461.4.patch, HBASE-5186.D1461.5.patch, HBASE-5186.D1461.6.patch, 
 HBASE-5186.D1461.7.patch, HBASE-5186.D1461.8.patch


 It will be useful to have some metrics (queue length, waiting time, 
 processing time ...) similar to Hadoop RPC server. This allows us to monitor 
 system health also provide a tool to diagnose the problem where thrift calls 
 are slow.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5200) AM.ProcessRegionInTransition() and AM.handleRegion() race thus leaving the region assignment inconsistent

2012-02-03 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199950#comment-13199950
 ] 

Hadoop QA commented on HBASE-5200:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12513158/5200-v2.txt
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

-1 javadoc.  The javadoc tool appears to have generated -136 warning 
messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 155 new Findbugs (version 
1.3.9) warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.TestInfoServers
  org.apache.hadoop.hbase.client.TestAdmin
  org.apache.hadoop.hbase.io.hfile.TestHFileBlock
  org.apache.hadoop.hbase.master.TestZKBasedOpenCloseRegion
  org.apache.hadoop.hbase.mapreduce.TestImportTsv
  org.apache.hadoop.hbase.mapred.TestTableMapReduce
  org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/898//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/898//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/898//console

This message is automatically generated.

 AM.ProcessRegionInTransition() and AM.handleRegion() race thus leaving the 
 region assignment inconsistent
 -

 Key: HBASE-5200
 URL: https://issues.apache.org/jira/browse/HBASE-5200
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.5
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.94.0, 0.90.7, 0.92.1

 Attachments: 5200-v2.txt, HBASE-5200.patch, HBASE-5200_1.patch, 
 TEST-org.apache.hadoop.hbase.master.TestRestartCluster.xml


 This is the scenario
 Consider a case where the balancer is going on thus trying to close regions 
 in a RS.
 Before we could close a master switch happens.  
 On Master switch the set of nodes that are in RIT is collected and we first 
 get Data and start watching the node
 After that the node data is added into RIT.
 Now by this time (before adding to RIT) if the RS to which close was called 
 does a transition in AM.handleRegion() we miss the handling saying RIT state 
 was null.
 {code}
 2012-01-13 10:50:46,358 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 a66d281d231dfcaea97c270698b26b6f from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,358 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 c12e53bfd48ddc5eec507d66821c4d23 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,358 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 59ae13de8c1eb325a0dd51f4902d2052 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 f45bc9614d7575f35244849af85aa078 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 cc3ecd7054fe6cd4a1159ed92fd62641 from server 
 HOST-192-168-47-204,20020,1326342744518 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 3af40478a17fee96b4a192b22c90d5a2 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received

[jira] [Commented] (HBASE-5229) Explore building blocks for multi-row local transactions.


[ 
https://issues.apache.org/jira/browse/HBASE-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199953#comment-13199953
 ] 

jirapos...@reviews.apache.org commented on HBASE-5229:
--



bq.  On 2012-02-03 05:01:34, Jimmy Xiang wrote:
bq.   
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiRowMutation.java,
 line 67
bq.   https://reviews.apache.org/r/3748/diff/1/?file=72039#file72039line67
bq.  
bq.   Should we check they are for different rows here?
bq.  
bq.  Lars Hofhansl wrote:
bq.  It is still valid to pass the same rows here.

That's right.  Should the server side share the same function to do 
MultiRowMutation and RowMutation?  Looks the server side doesn't have much 
difference for mutateRow() and mutateRows()


- Jimmy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3748/#review4789
---


On 2012-02-03 01:29:58, Lars Hofhansl wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/3748/
bq.  ---
bq.  
bq.  (Updated 2012-02-03 01:29:58)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This builds on HBASE-3584, HBASE-5203, and HBASE-5304.
bq.  
bq.  Multiple Rows can be locked and applied atomically as long as the 
application ensures that all rows reside in the same Region (by presplitting or 
a custom RegionSplitPolicy).
bq.  At SFDC we can use this to colocate subsets of a tenant's data and allow 
atomic operations over these subsets.
bq.  
bq.  Obviously this is an advanced features and this prominently called out in 
the Javadoc.
bq.  
bq.  
bq.  This addresses bug HBASE-5229.
bq.  https://issues.apache.org/jira/browse/HBASE-5229
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiRowMutation.java
 PRE-CREATION 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java
 1239953 
bq.  
bq.  Diff: https://reviews.apache.org/r/3748/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  Tests added to TestFromClientSide and TestAtomicOperation
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Lars
bq.  
bq.



 Explore building blocks for multi-row local transactions.
 ---

 Key: HBASE-5229
 URL: https://issues.apache.org/jira/browse/HBASE-5229
 Project: HBase
  Issue Type: New Feature
  Components: client, regionserver
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.0

 Attachments: 5229-multiRow.txt, 5229-seekto-v2.txt, 5229-seekto.txt, 
 5229.txt


 HBase should provide basic building blocks for multi-row local transactions. 
 Local means that we do this by co-locating the data. Global (cross region) 
 transactions are not discussed here.
 After a bit of discussion two solutions have emerged:
 1. Keep the row-key for determining grouping and location and allow efficient 
 intra-row scanning. A client application would then model tables as 
 HBase-rows.
 2. Define a prefix-length in HTableDescriptor that defines a grouping of 
 rows. Regions will then never be split inside a grouping

[jira] [Commented] (HBASE-5128) [uber hbck] Enable hbck to automatically repair table integrity problems as well as region consistency problems while online.

2012-02-03 Thread Jonathan Hsieh (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199955#comment-13199955
]

Jonathan Hsieh commented on HBASE-5128:
---

Update:

Recently found a case that may have been suffering from parent region not
getting removed by the catalog janitor. Since we rely on hdfs being ground
truth and this version did not check have offline/split status in meta, this
resulted in the tool attempting to merge all regions into one mega region.
Harsh mentioned, that the parent region cleanup issue might be related to
(HBASE-4799) (target cluster didn't have this patch).

Next cuts will add some failsafes -- specifiers to repair only specific tables
and to skip if a merge attempts to merge more than a specified number of
regions into one region.

Also, at the moment, I also have first cut versions that for 0.92/trunk but
have one flaky test.

[uber hbck] Enable hbck to automatically repair table integrity problems as
well as region consistency problems while online.
-

Key: HBASE-5128
URL: https://issues.apache.org/jira/browse/HBASE-5128
Project: HBase
Issue Type: New Feature
Components: hbck
Affects Versions: 0.90.5, 0.92.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh

The current (0.90.5, 0.92.0rc2) versions of hbck detects most of region
consistency and table integrity invariant violations. However with '-fix' it
can only automatically repair region consistency cases having to do with
deployment problems. This updated version should be able to handle all cases
(including a new orphan regiondir case). When complete will likely deprecate
the OfflineMetaRepair tool and subsume several open META-hole related issue.
Here's the approach (from the comment of at the top of the new version of the
file).
{code}
/**
* HBaseFsck (hbck) is a tool for checking and repairing region consistency
and
* table integrity.
*
* Region consistency checks verify that META, region deployment on
* region servers and the state of data in HDFS (.regioninfo files) all are in
* accordance.
*
* Table integrity checks verify that that all possible row keys can resolve
to
* exactly one region of a table. This means there are no individual
degenerate
* or backwards regions; no holes between regions; and that there no
overlapping
* regions.
*
* The general repair strategy works in these steps.
* 1) Repair Table Integrity on HDFS. (merge or fabricate regions)
* 2) Repair Region Consistency with META and assignments
*
* For table integrity repairs, the tables their region directories are
scanned
* for .regioninfo files. Each table's integrity is then verified. If there
* are any orphan regions (regions with no .regioninfo files), or holes, new
* regions are fabricated. Backwards regions are sidelined as well as empty
* degenerate (endkey==startkey) regions. If there are any overlapping
regions,
* a new region is created and all data is merged into the new region.
*
* Table integrity repairs deal solely with HDFS and can be done offline --
the
* hbase region servers or master do not need to be running. These phase can
be
* use to completely reconstruct the META table in an offline fashion.
*
* Region consistency requires three conditions -- 1) valid .regioninfo file
* present in an hdfs region dir, 2) valid row with .regioninfo data in META,
* and 3) a region is deployed only at the regionserver that is was assigned
to.
*
* Region consistency requires hbck to contact the HBase master and region
* servers, so the connect() must first be called successfully. Much of the
* region consistency information is transient and less risky to repair.
*/
{code}

[jira] [Commented] (HBASE-5128) [uber hbck] Enable hbck to automatically repair table integrity problems as well as region consistency problems while online.

2012-02-03 Thread Jonathan Hsieh (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199967#comment-13199967
]

Jonathan Hsieh commented on HBASE-5128:
---

Jimmy mentions this actually may be HBASE-4238.

[uber hbck] Enable hbck to automatically repair table integrity problems as
well as region consistency problems while online.
-

[jira] [Commented] (HBASE-5318) Support Eclipse Indigo

[
https://issues.apache.org/jira/browse/HBASE-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199974#comment-13199974
]

Zhihong Yu commented on HBASE-5318:
---

I don't see a QA test run in this JIRA.

Support Eclipse Indigo
---

[jira] [Commented] (HBASE-5330) TestCompactSelection - adding 2 test cases to testCompactionRatio

[
https://issues.apache.org/jira/browse/HBASE-5330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199977#comment-13199977
]

Hadoop QA commented on HBASE-5330:
--

-1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12513160/TestCompactSelection_hbase_5330.java.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

-1 javadoc. The javadoc tool appears to have generated -136 warning
messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 154 new Findbugs (version
1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.io.hfile.TestHFileBlock
org.apache.hadoop.hbase.TestInfoServers
org.apache.hadoop.hbase.master.TestZKBasedOpenCloseRegion

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/899//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/899//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/899//console

This message is automatically generated.

TestCompactSelection - adding 2 test cases to testCompactionRatio
-

Key: HBASE-5330
URL: https://issues.apache.org/jira/browse/HBASE-5330
Project: HBase
Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
Attachments: TestCompactSelection_hbase_5330.java.patch

There were three existing assertions in TestCompactSelection
testCompactionRatio that did max # of files assertions...
{code}
assertEquals(maxFiles,

store.compactSelection(sfCreate(7,6,5,4,3,2,1)).getFilesToCompact().size());
{code}
... and for references ...
{code}
assertEquals(maxFiles,
store.compactSelection(sfCreate(true,
7,6,5,4,3,2,1)).getFilesToCompact().size());
{code}

... but they didn't assert against which StoreFiles got selected. While the
number of StoreFiles is the same, the files selected are actually different,
and I thought that there should be explicit assertions showing that.

[jira] [Reopened] (HBASE-5318) Support Eclipse Indigo

2012-02-03 Thread Zhihong Yu (Reopened) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Zhihong Yu reopened HBASE-5318:
---

After reverting this patch, TestInfoServers passed.

Support Eclipse Indigo
---

[jira] [Commented] (HBASE-5318) Support Eclipse Indigo

[
https://issues.apache.org/jira/browse/HBASE-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1315#comment-1315
]

Jesse Yates commented on HBASE-5318:

That is really odd...TestInfoServers shouldn't at all be affected by this
patch. I'll run it locally and see.

Support Eclipse Indigo
---

[jira] [Commented] (HBASE-5329) addRowLock() may allocate duplicate lock id, causing the client to be blocked

2012-02-03 Thread Ming Ma (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1316#comment-1316
 ] 

Ming Ma commented on HBASE-5329:


It seems addScanner() might have similar issue.

 addRowLock() may allocate duplicate lock id, causing the client to be blocked
 -

 Key: HBASE-5329
 URL: https://issues.apache.org/jira/browse/HBASE-5329
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.3
 Environment: Red Hat Enterprise Linux Server release 5.4 
Reporter: liaoxiangui
Priority: Critical

 {code}
 protected long addRowLock(Integer r, HRegion region) throws 
 LeaseStillHeldException
 {
   long lockId = -1L;
   lockId = rand.nextLong();   //!!!may generate duplicated 
 id，bug?
   String lockName = String.valueOf(lockId);
   rowlocks.put(lockName, r);
   this.leases.createLease(lockName, new RowLockListener(lockName, 
 region));
   return lockId;
 }
 {code}
 In addRowLock(),rand may generate duplicated lock id, it may cause 
 regionserver throw exception(Leases$LeaseStillHeldException).The client will 
 be blocked until old rowlock is released.
 {code}
 2012-02-03 15:21:50,084 ERROR 
 org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock 
 (fsOk: true)
 org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException
 at 
 org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:1986)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:1963)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5304) Pluggable split key policy


[ 
https://issues.apache.org/jira/browse/HBASE-5304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1319#comment-1319
 ] 

Hudson commented on HBASE-5304:
---

Integrated in HBase-TRUNK #2653 (See 
[https://builds.apache.org/job/HBase-TRUNK/2653/])
HBASE-5304 Pluggable split key policy

larsh : 
Files : 
* /hbase/trunk/src/docbkx/book.xml
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ConstantSizeRegionSplitPolicy.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionSplitPolicy.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/PrefixSplitKeyPolicy.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionSplitPolicy.java


 Pluggable split key policy
 --

 Key: HBASE-5304
 URL: https://issues.apache.org/jira/browse/HBASE-5304
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.0

 Attachments: 5304-v4.txt, 5304-v5.txt, 5304-v6.txt, 5304-v7.txt


 We need a way to specify custom policies to determine split keys.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4658) Put attributes are not exposed via the ThriftServer


[ 
https://issues.apache.org/jira/browse/HBASE-4658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1318#comment-1318
 ] 

Hudson commented on HBASE-4658:
---

Integrated in HBase-TRUNK #2653 (See 
[https://builds.apache.org/job/HBase-TRUNK/2653/])
HBASE-4658 Put attributes are not exposed via the ThriftServer (Dhruba)

tedyu : 
Files : 
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/thrift/generated/Hbase.java
* /hbase/trunk/src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java


 Put attributes are not exposed via the ThriftServer
 ---

 Key: HBASE-4658
 URL: https://issues.apache.org/jira/browse/HBASE-4658
 Project: HBase
  Issue Type: Bug
  Components: thrift
Reporter: dhruba borthakur
Assignee: dhruba borthakur
 Attachments: D1563.1.patch, D1563.1.patch, D1563.1.patch, 
 D1563.2.patch, D1563.2.patch, D1563.2.patch, D1563.3.patch, D1563.3.patch, 
 D1563.3.patch, ThriftPutAttributes1.txt


 The Put api also takes in a bunch of arbitrary attributes that an application 
 can use to associate metadata with each put operation. This is not exposed 
 via Thrift.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5318) Support Eclipse Indigo

2012-02-03 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1320#comment-1320
 ] 

Hudson commented on HBASE-5318:
---

Integrated in HBase-TRUNK #2653 (See 
[https://builds.apache.org/job/HBase-TRUNK/2653/])
HBASE-5318 Support Eclipse Indigo (Jesse Yates)

larsh : 
Files : 
* /hbase/trunk/pom.xml


 Support Eclipse Indigo 
 ---

 Key: HBASE-5318
 URL: https://issues.apache.org/jira/browse/HBASE-5318
 Project: HBase
  Issue Type: Improvement
  Components: build
Affects Versions: 0.94.0
 Environment: Eclipse Indigo (1.4.1) which includes m2eclipse (1.0 
 SR1).
Reporter: Jesse Yates
Assignee: Jesse Yates
Priority: Minor
  Labels: maven
 Attachments: mvn_HBASE-5318_r0.patch


 The current 'standard' release of Eclipse (indigo) comes with m2eclipse 
 installed. However, as of m2e v1.0, interesting lifecycle phases are now 
 handled via a 'connector'. However, several of the plugins we use don't 
 support connectors. This means that eclipse bails out and won't build the 
 project or view it as 'working' even though it builds just fine from the the 
 command line.
 Since Eclipse is one of the major java IDEs and that Indigo has been around 
 for a while, we should make it easy to for new devs to pick up the code and 
 for older devs to upgrade painlessly. The original build should not be 
 modified in any significant way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5229) Explore building blocks for multi-row local transactions.

[
https://issues.apache.org/jira/browse/HBASE-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1322#comment-1322
]

jirapos...@reviews.apache.org commented on HBASE-5229:
--

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3748/
---

(Updated 2012-02-03 19:59:55.515243)

Review request for hbase.

Changes
---

Addressed Ted's and Jimmy's comments.
Also:
* RowMutation and MultiRowMutation are evolved independently (at the expense of
some replicated code)
* MultiRowMutation cannot participate itself in a Multi operation (because it
does not implement Row). Could be made working, but would need some refactoring
in a different jira.

Summary
---

This builds on HBASE-3584, HBASE-5203, and HBASE-5304.

Multiple Rows can be locked and applied atomically as long as the application
ensures that all rows reside in the same Region (by presplitting or a custom
RegionSplitPolicy).
At SFDC we can use this to colocate subsets of a tenant's data and allow atomic
operations over these subsets.

Obviously this is an advanced features and this prominently called out in the
Javadoc.

This addresses bug HBASE-5229.
https://issues.apache.org/jira/browse/HBASE-5229

Diffs (updated)
-

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java
1239953

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java
1239953

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiRowMutation.java
PRE-CREATION

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
1239953

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
1239953

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
1239953

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
1239953

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
1239953

http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
1239953

http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
1239953

http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java
1239953

Diff: https://reviews.apache.org/r/3748/diff

Testing
---

Tests added to TestFromClientSide and TestAtomicOperation

Thanks,

Lars

Explore building blocks for multi-row local transactions.
---

Attachments: 5229-multiRow.txt, 5229-seekto-v2.txt, 5229-seekto.txt,
5229.txt

[jira] [Commented] (HBASE-5318) Support Eclipse Indigo


[ 
https://issues.apache.org/jira/browse/HBASE-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1325#comment-1325
 ] 

Zhihong Yu commented on HBASE-5318:
---

Here is the exception that prevented Master from starting:
{code}
master.HMaster(1437): Unhandled exception. Starting shutdown.
java.io.FileNotFoundException: hbase-webapps/master not found in CLASSPATH
  at org.apache.hadoop.hbase.util.InfoServer.getWebAppsPath(InfoServer.java:108)
  at org.apache.hadoop.hbase.util.InfoServer.getWebAppsPath(InfoServer.java:125)
  at org.apache.hadoop.http.HttpServer.lt;initgt;(HttpServer.java:159)
  at org.apache.hadoop.http.HttpServer.lt;initgt;(HttpServer.java:113)
  at org.apache.hadoop.hbase.util.InfoServer.lt;initgt;(InfoServer.java:60)
  at 
org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:784)
  at 
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:499)
  at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:348)
  at java.lang.Thread.run(Thread.java:680)
{code}

@Lars:
Please revert this patch.

When a new patch comes, we should run it through Hadoop QA first.

 Support Eclipse Indigo 
 ---

 Key: HBASE-5318
 URL: https://issues.apache.org/jira/browse/HBASE-5318
 Project: HBase
  Issue Type: Improvement
  Components: build
Affects Versions: 0.94.0
 Environment: Eclipse Indigo (1.4.1) which includes m2eclipse (1.0 
 SR1).
Reporter: Jesse Yates
Assignee: Jesse Yates
Priority: Minor
  Labels: maven
 Attachments: mvn_HBASE-5318_r0.patch


 The current 'standard' release of Eclipse (indigo) comes with m2eclipse 
 installed. However, as of m2e v1.0, interesting lifecycle phases are now 
 handled via a 'connector'. However, several of the plugins we use don't 
 support connectors. This means that eclipse bails out and won't build the 
 project or view it as 'working' even though it builds just fine from the the 
 command line.
 Since Eclipse is one of the major java IDEs and that Indigo has been around 
 for a while, we should make it easy to for new devs to pick up the code and 
 for older devs to upgrade painlessly. The original build should not be 
 modified in any significant way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5318) Support Eclipse Indigo

[
https://issues.apache.org/jira/browse/HBASE-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1329#comment-1329
]

Lars Hofhansl commented on HBASE-5318:
--

I agree... Will revert until we figure out what the problem is.

Support Eclipse Indigo
---

[jira] [Commented] (HBASE-5318) Support Eclipse Indigo

[
https://issues.apache.org/jira/browse/HBASE-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200012#comment-13200012
]

Lars Hofhansl commented on HBASE-5318:
--

Reverted.

Support Eclipse Indigo
---

[jira] [Updated] (HBASE-5292) getsize per-CF metric incorrectly counts compaction related reads as well

2012-02-03 Thread Phabricator (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5292:
---

Attachment: D1527.4.patch

zhiqiu updated the revision [jira] [HBASE-5292] [89-fb] Prevent counting 
getSize on compactions.
Reviewers: Kannan, Liyin, JIRA

  Fixed a bug that the two tests in TestRegionServerMetrics cannot be run 
together

REVISION DETAIL
  https://reviews.facebook.net/D1527

AFFECTED FILES
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
  src/main/java/org/apache/hadoop/hbase/regionserver/InternalScanner.java
  src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueHeap.java
  src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
  
src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java


 getsize per-CF metric incorrectly counts compaction related reads as well 
 --

 Key: HBASE-5292
 URL: https://issues.apache.org/jira/browse/HBASE-5292
 Project: HBase
  Issue Type: Bug
Reporter: Kannan Muthukkaruppan
 Attachments: D1527.1.patch, D1527.2.patch, D1527.3.patch, 
 D1527.4.patch


 The per-CF getsize metric's intent was to track bytes returned (to HBase 
 clients) per-CF. [Note: We already have metrics to track # of HFileBlock's 
 read for compaction vs. non-compaction cases -- e.g., compactionblockreadcnt 
 vs. fsblockreadcnt.]
 Currently, the getsize metric gets updated for both client initiated 
 Get/Scan operations as well for compaction related reads. The metric is 
 updated in StoreScanner.java:next() when the Scan query matcher returns an 
 INCLUDE* code via a:
  HRegion.incrNumericMetric(this.metricNameGetsize, copyKv.getLength());
 We should not do the above in case of compactions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5318) Support Eclipse Indigo

2012-02-03 Thread stack (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200014#comment-13200014
 ] 

stack commented on HBASE-5318:
--

That looks like CLASSPATH messup where hadoop jar is now before hbase jar 
(classloader is finding a class that is in hadoop and hbase jar in the hadoop 
jar and then doing subsequent lookups in hadoop jar)

 Support Eclipse Indigo 
 ---

 Key: HBASE-5318
 URL: https://issues.apache.org/jira/browse/HBASE-5318
 Project: HBase
  Issue Type: Improvement
  Components: build
Affects Versions: 0.94.0
 Environment: Eclipse Indigo (1.4.1) which includes m2eclipse (1.0 
 SR1).
Reporter: Jesse Yates
Assignee: Jesse Yates
Priority: Minor
  Labels: maven
 Attachments: mvn_HBASE-5318_r0.patch


 The current 'standard' release of Eclipse (indigo) comes with m2eclipse 
 installed. However, as of m2e v1.0, interesting lifecycle phases are now 
 handled via a 'connector'. However, several of the plugins we use don't 
 support connectors. This means that eclipse bails out and won't build the 
 project or view it as 'working' even though it builds just fine from the the 
 command line.
 Since Eclipse is one of the major java IDEs and that Indigo has been around 
 for a while, we should make it easy to for new devs to pick up the code and 
 for older devs to upgrade painlessly. The original build should not be 
 modified in any significant way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5325) Expose basic information about the master-status through jmx beans

2012-02-03 Thread Enis Soztutar (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200015#comment-13200015
 ] 

Enis Soztutar commented on HBASE-5325:
--

@Stack, 
I think, what you are suggesting for registering multiple MBeans, and 
co-processor Mbeans can be best handled after we migrated to the metrics2 
system (HBASE-4050), no?. I agree that master/rs mbeans incorporate more 
functionality that just metrics, but we can start with the metrics aggregation 
design and go from there.

 Expose basic information about the master-status through jmx beans 
 ---

 Key: HBASE-5325
 URL: https://issues.apache.org/jira/browse/HBASE-5325
 Project: HBase
  Issue Type: Bug
Reporter: Hitesh Shah
Priority: Minor
 Attachments: HBASE-5325.wip.patch


 Similar to the Namenode and Jobtracker, it would be good if the hbase master 
 could expose some information through mbeans.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5331) Off by one bug in util.HMerge

2012-02-03 Thread Sandy Pratt (Created) (JIRA)

Off by one bug in util.HMerge
-

 Key: HBASE-5331
 URL: https://issues.apache.org/jira/browse/HBASE-5331
 Project: HBase
  Issue Type: Bug
  Components: util
Affects Versions: 0.90.1
 Environment: NA
Reporter: Sandy Pratt
 Fix For: 0.94.0


It looks like there's an off by one bug in the OfflineMerger constructor in 
util.HMerge:

  InternalScanner rootScanner =
root.getScanner(scan);

  try {
ListKeyValue results = new ArrayListKeyValue();
while(rootScanner.next(results)) {
  for(KeyValue kv: results) {
HRegionInfo info = Writables.getHRegionInfoOrNull(kv.getValue());
if (info != null) {
  metaRegions.add(info);
}
  }
}
  } finally {
...
  }

That call to InternalScanner.next() in the while condition returns true if 
there's another result *after* the one it just loaded into the out param.  That 
is, after it reads the last row into the 'results' collection, it returns false 
and the loop exits with that last row unread.  It probably wants to be 
structured more like this:

final InternalScanner metaScanner = meta.getScanner(scan); ListKeyValue 
results = Lists.newArrayList();

while (true) {

boolean hasMore = metaScanner.next(results);

for (KeyValue kv : results) {
HRegionInfo hri = Writables.getHRegionInfoOrNull(kv.getValue());
if (hri != null) {
regionInfo.add(hri);
}
}

if (!hasMore) {
break;
}
}

The loop in util.HMerge is scanning ROOT for META regions.  So this bug will 
only be hit when there is more than one region of META.  Personally, I don't 
have any installations with more than one META region, and I'm not sure if 
anyone does, so this might be a moot point.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5331) Off by one bug in util.HMerge

2012-02-03 Thread Sandy Pratt (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200019#comment-13200019
 ] 

Sandy Pratt commented on HBASE-5331:


Woops, looks like this is already fixed on trunk!  Feel free to close this...

 Off by one bug in util.HMerge
 -

 Key: HBASE-5331
 URL: https://issues.apache.org/jira/browse/HBASE-5331
 Project: HBase
  Issue Type: Bug
  Components: util
Affects Versions: 0.90.1
 Environment: NA
Reporter: Sandy Pratt
 Fix For: 0.94.0


 It looks like there's an off by one bug in the OfflineMerger constructor in 
 util.HMerge:
   InternalScanner rootScanner =
 root.getScanner(scan);
   try {
 ListKeyValue results = new ArrayListKeyValue();
 while(rootScanner.next(results)) {
   for(KeyValue kv: results) {
 HRegionInfo info = Writables.getHRegionInfoOrNull(kv.getValue());
 if (info != null) {
   metaRegions.add(info);
 }
   }
 }
   } finally {
   ...
   }
 That call to InternalScanner.next() in the while condition returns true if 
 there's another result *after* the one it just loaded into the out param.  
 That is, after it reads the last row into the 'results' collection, it 
 returns false and the loop exits with that last row unread.  It probably 
 wants to be structured more like this:
 final InternalScanner metaScanner = meta.getScanner(scan); ListKeyValue 
 results = Lists.newArrayList();
 while (true) {
   boolean hasMore = metaScanner.next(results);
   for (KeyValue kv : results) {
   HRegionInfo hri = Writables.getHRegionInfoOrNull(kv.getValue());
   if (hri != null) {
   regionInfo.add(hri);
   }
   }
   
   if (!hasMore) {
   break;
   }
 }
 The loop in util.HMerge is scanning ROOT for META regions.  So this bug will 
 only be hit when there is more than one region of META.  Personally, I don't 
 have any installations with more than one META region, and I'm not sure if 
 anyone does, so this might be a moot point.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-5331) Off by one bug in util.HMerge

2012-02-03 Thread Sandy Pratt (Resolved) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sandy Pratt resolved HBASE-5331.


Resolution: Not A Problem

Already fixed on trunk

 Off by one bug in util.HMerge
 -

 Key: HBASE-5331
 URL: https://issues.apache.org/jira/browse/HBASE-5331
 Project: HBase
  Issue Type: Bug
  Components: util
Affects Versions: 0.90.1
 Environment: NA
Reporter: Sandy Pratt
 Fix For: 0.94.0


 It looks like there's an off by one bug in the OfflineMerger constructor in 
 util.HMerge:
   InternalScanner rootScanner =
 root.getScanner(scan);
   try {
 ListKeyValue results = new ArrayListKeyValue();
 while(rootScanner.next(results)) {
   for(KeyValue kv: results) {
 HRegionInfo info = Writables.getHRegionInfoOrNull(kv.getValue());
 if (info != null) {
   metaRegions.add(info);
 }
   }
 }
   } finally {
   ...
   }
 That call to InternalScanner.next() in the while condition returns true if 
 there's another result *after* the one it just loaded into the out param.  
 That is, after it reads the last row into the 'results' collection, it 
 returns false and the loop exits with that last row unread.  It probably 
 wants to be structured more like this:
 final InternalScanner metaScanner = meta.getScanner(scan); ListKeyValue 
 results = Lists.newArrayList();
 while (true) {
   boolean hasMore = metaScanner.next(results);
   for (KeyValue kv : results) {
   HRegionInfo hri = Writables.getHRegionInfoOrNull(kv.getValue());
   if (hri != null) {
   regionInfo.add(hri);
   }
   }
   
   if (!hasMore) {
   break;
   }
 }
 The loop in util.HMerge is scanning ROOT for META regions.  So this bug will 
 only be hit when there is more than one region of META.  Personally, I don't 
 have any installations with more than one META region, and I'm not sure if 
 anyone does, so this might be a moot point.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5325) Expose basic information about the master-status through jmx beans


[ 
https://issues.apache.org/jira/browse/HBASE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200034#comment-13200034
 ] 

Zhihong Yu commented on HBASE-5325:
---

HADOOP-6728 which is referred to by HBASE-4050 has been dormant for more than 
half year.

 Expose basic information about the master-status through jmx beans 
 ---

 Key: HBASE-5325
 URL: https://issues.apache.org/jira/browse/HBASE-5325
 Project: HBase
  Issue Type: Bug
Reporter: Hitesh Shah
Priority: Minor
 Attachments: HBASE-5325.wip.patch


 Similar to the Namenode and Jobtracker, it would be good if the hbase master 
 could expose some information through mbeans.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5200) AM.ProcessRegionInTransition() and AM.handleRegion() race thus leaving the region assignment inconsistent


 [ 
https://issues.apache.org/jira/browse/HBASE-5200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5200:
--

Attachment: (was: 5200-v2.txt)

 AM.ProcessRegionInTransition() and AM.handleRegion() race thus leaving the 
 region assignment inconsistent
 -

 Key: HBASE-5200
 URL: https://issues.apache.org/jira/browse/HBASE-5200
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.5
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.94.0, 0.90.7, 0.92.1

 Attachments: 5200-v2.txt, HBASE-5200.patch, HBASE-5200_1.patch, 
 TEST-org.apache.hadoop.hbase.master.TestRestartCluster.xml


 This is the scenario
 Consider a case where the balancer is going on thus trying to close regions 
 in a RS.
 Before we could close a master switch happens.  
 On Master switch the set of nodes that are in RIT is collected and we first 
 get Data and start watching the node
 After that the node data is added into RIT.
 Now by this time (before adding to RIT) if the RS to which close was called 
 does a transition in AM.handleRegion() we miss the handling saying RIT state 
 was null.
 {code}
 2012-01-13 10:50:46,358 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 a66d281d231dfcaea97c270698b26b6f from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,358 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 c12e53bfd48ddc5eec507d66821c4d23 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,358 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 59ae13de8c1eb325a0dd51f4902d2052 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 f45bc9614d7575f35244849af85aa078 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 cc3ecd7054fe6cd4a1159ed92fd62641 from server 
 HOST-192-168-47-204,20020,1326342744518 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 3af40478a17fee96b4a192b22c90d5a2 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 e6096a8466e730463e10d3d61f809b92 from server 
 HOST-192-168-47-204,20020,1326342744518 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 4806781a1a23066f7baed22b4d237e24 from server 
 HOST-192-168-47-204,20020,1326342744518 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 d69e104131accaefe21dcc01fddc7629 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 {code}
 In branch the CLOSING node is created by RS thus leading to more 
 inconsistency.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5200) AM.ProcessRegionInTransition() and AM.handleRegion() race thus leaving the region assignment inconsistent


 [ 
https://issues.apache.org/jira/browse/HBASE-5200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5200:
--

Attachment: 5200-v2.txt

Re-attach patch v2 for Hadoop QA.

 AM.ProcessRegionInTransition() and AM.handleRegion() race thus leaving the 
 region assignment inconsistent
 -

 Key: HBASE-5200
 URL: https://issues.apache.org/jira/browse/HBASE-5200
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.5
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.94.0, 0.90.7, 0.92.1

 Attachments: 5200-v2.txt, HBASE-5200.patch, HBASE-5200_1.patch, 
 TEST-org.apache.hadoop.hbase.master.TestRestartCluster.xml


 This is the scenario
 Consider a case where the balancer is going on thus trying to close regions 
 in a RS.
 Before we could close a master switch happens.  
 On Master switch the set of nodes that are in RIT is collected and we first 
 get Data and start watching the node
 After that the node data is added into RIT.
 Now by this time (before adding to RIT) if the RS to which close was called 
 does a transition in AM.handleRegion() we miss the handling saying RIT state 
 was null.
 {code}
 2012-01-13 10:50:46,358 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 a66d281d231dfcaea97c270698b26b6f from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,358 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 c12e53bfd48ddc5eec507d66821c4d23 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,358 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 59ae13de8c1eb325a0dd51f4902d2052 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 f45bc9614d7575f35244849af85aa078 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 cc3ecd7054fe6cd4a1159ed92fd62641 from server 
 HOST-192-168-47-204,20020,1326342744518 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 3af40478a17fee96b4a192b22c90d5a2 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 e6096a8466e730463e10d3d61f809b92 from server 
 HOST-192-168-47-204,20020,1326342744518 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 4806781a1a23066f7baed22b4d237e24 from server 
 HOST-192-168-47-204,20020,1326342744518 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 d69e104131accaefe21dcc01fddc7629 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 {code}
 In branch the CLOSING node is created by RS thus leading to more 
 inconsistency.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5229) Explore building blocks for multi-row local transactions.

[
https://issues.apache.org/jira/browse/HBASE-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200059#comment-13200059
]

Zhihong Yu commented on HBASE-5229:
---

Latest patch looks much cleaner.
Shall we let Hadoop QA run over the patch ?

Explore building blocks for multi-row local transactions.
---

Attachments: 5229-multiRow.txt, 5229-seekto-v2.txt, 5229-seekto.txt,
5229.txt

[jira] [Assigned] (HBASE-5329) addRowLock() may allocate duplicate lock id, causing the client to be blocked

2012-02-03 Thread Zhihong Yu (Assigned) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu reassigned HBASE-5329:
-

Assignee: Zhihong Yu

 addRowLock() may allocate duplicate lock id, causing the client to be blocked
 -

 Key: HBASE-5329
 URL: https://issues.apache.org/jira/browse/HBASE-5329
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.3
 Environment: Red Hat Enterprise Linux Server release 5.4 
Reporter: liaoxiangui
Assignee: Zhihong Yu
Priority: Critical

 {code}
 protected long addRowLock(Integer r, HRegion region) throws 
 LeaseStillHeldException
 {
   long lockId = -1L;
   lockId = rand.nextLong();   //!!!may generate duplicated 
 id，bug?
   String lockName = String.valueOf(lockId);
   rowlocks.put(lockName, r);
   this.leases.createLease(lockName, new RowLockListener(lockName, 
 region));
   return lockId;
 }
 {code}
 In addRowLock(),rand may generate duplicated lock id, it may cause 
 regionserver throw exception(Leases$LeaseStillHeldException).The client will 
 be blocked until old rowlock is released.
 {code}
 2012-02-03 15:21:50,084 ERROR 
 org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock 
 (fsOk: true)
 org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException
 at 
 org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:1986)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:1963)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5200) AM.ProcessRegionInTransition() and AM.handleRegion() race thus leaving the region assignment inconsistent

2012-02-03 Thread Lars Hofhansl (Issue Comment Edited) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5200:
--

Comment: was deleted

(was: -1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12513158/5200-v2.txt
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

-1 javadoc.  The javadoc tool appears to have generated -136 warning 
messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 155 new Findbugs (version 
1.3.9) warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.TestInfoServers
  org.apache.hadoop.hbase.client.TestAdmin
  org.apache.hadoop.hbase.io.hfile.TestHFileBlock
  org.apache.hadoop.hbase.master.TestZKBasedOpenCloseRegion
  org.apache.hadoop.hbase.mapreduce.TestImportTsv
  org.apache.hadoop.hbase.mapred.TestTableMapReduce
  org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/898//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/898//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/898//console

This message is automatically generated.)

 AM.ProcessRegionInTransition() and AM.handleRegion() race thus leaving the 
 region assignment inconsistent
 -

 Key: HBASE-5200
 URL: https://issues.apache.org/jira/browse/HBASE-5200
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.5
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.94.0, 0.90.7, 0.92.1

 Attachments: 5200-v2.txt, HBASE-5200.patch, HBASE-5200_1.patch, 
 TEST-org.apache.hadoop.hbase.master.TestRestartCluster.xml


 This is the scenario
 Consider a case where the balancer is going on thus trying to close regions 
 in a RS.
 Before we could close a master switch happens.  
 On Master switch the set of nodes that are in RIT is collected and we first 
 get Data and start watching the node
 After that the node data is added into RIT.
 Now by this time (before adding to RIT) if the RS to which close was called 
 does a transition in AM.handleRegion() we miss the handling saying RIT state 
 was null.
 {code}
 2012-01-13 10:50:46,358 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 a66d281d231dfcaea97c270698b26b6f from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,358 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 c12e53bfd48ddc5eec507d66821c4d23 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,358 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 59ae13de8c1eb325a0dd51f4902d2052 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 f45bc9614d7575f35244849af85aa078 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 cc3ecd7054fe6cd4a1159ed92fd62641 from server 
 HOST-192-168-47-204,20020,1326342744518 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 3af40478a17fee96b4a192b22c90d5a2 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region

[jira] [Commented] (HBASE-5292) getsize per-CF metric incorrectly counts compaction related reads as well

2012-02-03 Thread Phabricator (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200094#comment-13200094
]

Phabricator commented on HBASE-5292:

Kannan has commented on the revision [jira] [HBASE-5292] [89-fb] Prevent
counting getSize on compactions.

ok - looks good.

REVISION DETAIL
https://reviews.facebook.net/D1527

getsize per-CF metric incorrectly counts compaction related reads as well
--

Key: HBASE-5292
URL: https://issues.apache.org/jira/browse/HBASE-5292
Project: HBase
Issue Type: Bug
Reporter: Kannan Muthukkaruppan
Attachments: D1527.1.patch, D1527.2.patch, D1527.3.patch,
D1527.4.patch

The per-CF getsize metric's intent was to track bytes returned (to HBase
clients) per-CF. [Note: We already have metrics to track # of HFileBlock's
read for compaction vs. non-compaction cases -- e.g., compactionblockreadcnt
vs. fsblockreadcnt.]
Currently, the getsize metric gets updated for both client initiated
Get/Scan operations as well for compaction related reads. The metric is
updated in StoreScanner.java:next() when the Scan query matcher returns an
INCLUDE* code via a:
HRegion.incrNumericMetric(this.metricNameGetsize, copyKv.getLength());
We should not do the above in case of compactions.

[jira] [Updated] (HBASE-5318) Support Eclipse Indigo

2012-02-03 Thread Jesse Yates (Updated) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jesse Yates updated HBASE-5318:
---

Status: Patch Available (was: Reopened)

Support Eclipse Indigo
---

[jira] [Issue Comment Edited] (HBASE-5229) Explore building blocks for multi-row local transactions.


[ 
https://issues.apache.org/jira/browse/HBASE-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199940#comment-13199940
 ] 

Lars Hofhansl edited comment on HBASE-5229 at 2/3/12 10:27 PM:
---


bq.  On 2012-02-03 04:45:54, Ted Yu wrote:
bq.   
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java,
 line 96
bq.   https://reviews.apache.org/r/3748/diff/1/?file=72040#file72040line96
bq.  
bq.   version would be read / written twice, right ?
bq.  
bq.  Lars Hofhansl wrote:
bq.  Yes. Need to fix that.

And actually this is what I had in mind too. Both MultiRowMutation and 
RowMution write their version.
MultiRowMutation knowns to serialize mutations, that process needs to be 
versioned.
RowMutation adds some fields on top of that, those also need to be versioned.

Maybe the two should be entirely independent (at the expense of a few 
duplicated lines of code) so that they can evolve independently.


- Lars




  was (Author: jirapos...@reviews.apache.org):


bq.  On 2012-02-03 04:45:54, Ted Yu wrote:
bq.   
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java,
 line 96
bq.   https://reviews.apache.org/r/3748/diff/1/?file=72040#file72040line96
bq.  
bq.   version would be read / written twice, right ?
bq.  
bq.  Lars Hofhansl wrote:
bq.  Yes. Need to fix that.

And actually this is what I had in mind too. Both MultiRowMutation and 
RowMution write their version.
MultiRowMutation knowns to serialize mutations, that process needs to be 
versioned.
RowMutation adds some fields on top of that, those also need to be versioned.

Maybe the two should be entirely independent (at the expense of a few 
duplicated lines of code) so that they can evolve independently.


- Lars


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3748/#review4788
---


On 2012-02-03 01:29:58, Lars Hofhansl wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/3748/
bq.  ---
bq.  
bq.  (Updated 2012-02-03 01:29:58)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This builds on HBASE-3584, HBASE-5203, and HBASE-5304.
bq.  
bq.  Multiple Rows can be locked and applied atomically as long as the 
application ensures that all rows reside in the same Region (by presplitting or 
a custom RegionSplitPolicy).
bq.  At SFDC we can use this to colocate subsets of a tenant's data and allow 
atomic operations over these subsets.
bq.  
bq.  Obviously this is an advanced features and this prominently called out in 
the Javadoc.
bq.  
bq.  
bq.  This addresses bug HBASE-5229.
bq.  https://issues.apache.org/jira/browse/HBASE-5229
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiRowMutation.java
 PRE-CREATION 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java
 1239953 
bq.  
bq.  Diff: https://reviews.apache.org/r/3748/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  Tests added to TestFromClientSide and TestAtomicOperation
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Lars
bq.  
bq.


  
 Explore building

[jira] [Issue Comment Edited] (HBASE-5229) Explore building blocks for multi-row local transactions.

2012-02-03 Thread Lars Hofhansl (Issue Comment Edited) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199934#comment-13199934
 ] 

Lars Hofhansl edited comment on HBASE-5229 at 2/3/12 10:28 PM:
---


bq.  On 2012-02-03 04:45:54, Ted Yu wrote:
bq.   
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java,
 line 39
bq.   https://reviews.apache.org/r/3748/diff/1/?file=72040#file72040line39
bq.  
bq.   From its name, RowMutation seems to refer to single row. It is a 
little confusing RowMutation extends MultiRowMutation.
bq.  
bq.  Lars Hofhansl wrote:
bq.  Yeah. Maybe I'll have a common super class instead.

On 2nd thought. RowMutation *is* specialisation of MultiRowMutation, one that 
only allows a single row.


- Lars




  was (Author: jirapos...@reviews.apache.org):


bq.  On 2012-02-03 04:45:54, Ted Yu wrote:
bq.   
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java,
 line 39
bq.   https://reviews.apache.org/r/3748/diff/1/?file=72040#file72040line39
bq.  
bq.   From its name, RowMutation seems to refer to single row. It is a 
little confusing RowMutation extends MultiRowMutation.
bq.  
bq.  Lars Hofhansl wrote:
bq.  Yeah. Maybe I'll have a common super class instead.

On 2nd thought. RowMutation *is* specialisation of MultiRowMutation, one that 
only allows a single row.


- Lars


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3748/#review4788
---


On 2012-02-03 01:29:58, Lars Hofhansl wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/3748/
bq.  ---
bq.  
bq.  (Updated 2012-02-03 01:29:58)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This builds on HBASE-3584, HBASE-5203, and HBASE-5304.
bq.  
bq.  Multiple Rows can be locked and applied atomically as long as the 
application ensures that all rows reside in the same Region (by presplitting or 
a custom RegionSplitPolicy).
bq.  At SFDC we can use this to colocate subsets of a tenant's data and allow 
atomic operations over these subsets.
bq.  
bq.  Obviously this is an advanced features and this prominently called out in 
the Javadoc.
bq.  
bq.  
bq.  This addresses bug HBASE-5229.
bq.  https://issues.apache.org/jira/browse/HBASE-5229
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiRowMutation.java
 PRE-CREATION 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java
 1239953 
bq.  
bq.  Diff: https://reviews.apache.org/r/3748/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  Tests added to TestFromClientSide and TestAtomicOperation
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Lars
bq.  
bq.


  
 Explore building blocks for multi-row local transactions.
 ---

 Key: HBASE-5229
 URL: https://issues.apache.org/jira/browse/HBASE-5229
 Project: HBase
  Issue Type: New Feature
  Components: client, regionserver
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl

[jira] [Issue Comment Edited] (HBASE-5229) Explore building blocks for multi-row local transactions.

2012-02-03 Thread Lars Hofhansl (Issue Comment Edited) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199932#comment-13199932
 ] 

Lars Hofhansl edited comment on HBASE-5229 at 2/3/12 10:28 PM:
---


bq.  On 2012-02-03 05:01:34, Jimmy Xiang wrote:
bq.   
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiRowMutation.java,
 line 67
bq.   https://reviews.apache.org/r/3748/diff/1/?file=72039#file72039line67
bq.  
bq.   Should we check they are for different rows here?

It is still valid to pass the same rows here.


bq.  On 2012-02-03 05:01:34, Jimmy Xiang wrote:
bq.   
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java,
 line 46
bq.   https://reviews.apache.org/r/3748/diff/1/?file=72040#file72040line46
bq.  
bq.   This is called by default.  No need to call it explicitly.

rights... will fix.


- Lars


  was (Author: jirapos...@reviews.apache.org):


bq.  On 2012-02-03 05:01:34, Jimmy Xiang wrote:
bq.   
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiRowMutation.java,
 line 67
bq.   https://reviews.apache.org/r/3748/diff/1/?file=72039#file72039line67
bq.  
bq.   Should we check they are for different rows here?

It is still valid to pass the same rows here.


bq.  On 2012-02-03 05:01:34, Jimmy Xiang wrote:
bq.   
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java,
 line 46
bq.   https://reviews.apache.org/r/3748/diff/1/?file=72040#file72040line46
bq.  
bq.   This is called by default.  No need to call it explicitly.

rights... will fix.


- Lars


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3748/#review4789
---


On 2012-02-03 01:29:58, Lars Hofhansl wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/3748/
bq.  ---
bq.  
bq.  (Updated 2012-02-03 01:29:58)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This builds on HBASE-3584, HBASE-5203, and HBASE-5304.
bq.  
bq.  Multiple Rows can be locked and applied atomically as long as the 
application ensures that all rows reside in the same Region (by presplitting or 
a custom RegionSplitPolicy).
bq.  At SFDC we can use this to colocate subsets of a tenant's data and allow 
atomic operations over these subsets.
bq.  
bq.  Obviously this is an advanced features and this prominently called out in 
the Javadoc.
bq.  
bq.  
bq.  This addresses bug HBASE-5229.
bq.  https://issues.apache.org/jira/browse/HBASE-5229
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiRowMutation.java
 PRE-CREATION 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java
 1239953 
bq.  
bq.  Diff: https://reviews.apache.org/r/3748/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  Tests added to TestFromClientSide and TestAtomicOperation
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Lars
bq.  
bq.


  
 Explore building blocks for multi-row local transactions.
 ---

 Key:

[jira] [Issue Comment Edited] (HBASE-5229) Explore building blocks for multi-row local transactions.

2012-02-03 Thread Lars Hofhansl (Issue Comment Edited) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199523#comment-13199523
 ] 

Lars Hofhansl edited comment on HBASE-5229 at 2/3/12 10:28 PM:
---

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3748/#review4791
---



http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
https://reviews.apache.org/r/3748/#comment10547

My point was we shouldn't throw exception in this case.
MultiRowMutation should be delivered to the correct region.

I think we agree on the above.


- Ted


  was (Author: jirapos...@reviews.apache.org):

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3748/#review4791
---



http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
https://reviews.apache.org/r/3748/#comment10547

My point was we shouldn't throw exception in this case.
MultiRowMutation should be delivered to the correct region.

I think we agree on the above.


- Ted


On 2012-02-03 01:29:58, Lars Hofhansl wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/3748/
bq.  ---
bq.  
bq.  (Updated 2012-02-03 01:29:58)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This builds on HBASE-3584, HBASE-5203, and HBASE-5304.
bq.  
bq.  Multiple Rows can be locked and applied atomically as long as the 
application ensures that all rows reside in the same Region (by presplitting or 
a custom RegionSplitPolicy).
bq.  At SFDC we can use this to colocate subsets of a tenant's data and allow 
atomic operations over these subsets.
bq.  
bq.  Obviously this is an advanced features and this prominently called out in 
the Javadoc.
bq.  
bq.  
bq.  This addresses bug HBASE-5229.
bq.  https://issues.apache.org/jira/browse/HBASE-5229
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiRowMutation.java
 PRE-CREATION 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java
 1239953 
bq.  
bq.  Diff: https://reviews.apache.org/r/3748/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  Tests added to TestFromClientSide and TestAtomicOperation
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Lars
bq.  
bq.


  
 Explore building blocks for multi-row local transactions.
 ---

 Key: HBASE-5229
 URL: https://issues.apache.org/jira/browse/HBASE-5229
 Project: HBase
  Issue Type: New Feature
  Components: client, regionserver
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.0

 Attachments: 5229-multiRow.txt, 5229-seekto-v2.txt, 5229-seekto.txt, 
 5229.txt


 HBase should provide basic building blocks for multi-row local transactions. 
 Local means that we do this by co-locating the data. Global (cross region)

[jira] [Issue Comment Edited] (HBASE-5229) Explore building blocks for multi-row local transactions.

2012-02-03 Thread Lars Hofhansl (Issue Comment Edited) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199520#comment-13199520
 ] 

Lars Hofhansl edited comment on HBASE-5229 at 2/3/12 10:29 PM:
---

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3748/#review4789
---



http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiRowMutation.java
https://reviews.apache.org/r/3748/#comment10537

Should we check they are for different rows here?



http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java
https://reviews.apache.org/r/3748/#comment10543

This is called by default.  No need to call it explicitly.



http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java
https://reviews.apache.org/r/3748/#comment10545

This is called by default.  No need to call it explicitly.


- Jimmy


  was (Author: jirapos...@reviews.apache.org):

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3748/#review4789
---



http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiRowMutation.java
https://reviews.apache.org/r/3748/#comment10537

Should we check they are for different rows here?



http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java
https://reviews.apache.org/r/3748/#comment10543

This is called by default.  No need to call it explicitly.



http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java
https://reviews.apache.org/r/3748/#comment10545

This is called by default.  No need to call it explicitly.


- Jimmy


On 2012-02-03 01:29:58, Lars Hofhansl wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/3748/
bq.  ---
bq.  
bq.  (Updated 2012-02-03 01:29:58)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This builds on HBASE-3584, HBASE-5203, and HBASE-5304.
bq.  
bq.  Multiple Rows can be locked and applied atomically as long as the 
application ensures that all rows reside in the same Region (by presplitting or 
a custom RegionSplitPolicy).
bq.  At SFDC we can use this to colocate subsets of a tenant's data and allow 
atomic operations over these subsets.
bq.  
bq.  Obviously this is an advanced features and this prominently called out in 
the Javadoc.
bq.  
bq.  
bq.  This addresses bug HBASE-5229.
bq.  https://issues.apache.org/jira/browse/HBASE-5229
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiRowMutation.java
 PRE-CREATION 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java
 1239953 
bq.  
bq.  Diff: https://reviews.apache.org/r/3748/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  Tests added to TestFromClientSide and TestAtomicOperation
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Lars
bq.  
bq.


  
 Explore building

[jira] [Issue Comment Edited] (HBASE-5229) Explore building blocks for multi-row local transactions.

2012-02-03 Thread Lars Hofhansl (Issue Comment Edited) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13199933#comment-13199933
 ] 

Lars Hofhansl edited comment on HBASE-5229 at 2/3/12 10:28 PM:
---


bq.  On 2012-02-03 05:07:24, Ted Yu wrote:
bq.   
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java,
 line 3160
bq.   https://reviews.apache.org/r/3748/diff/1/?file=72045#file72045line3160
bq.  
bq.   My point was we shouldn't throw exception in this case.
bq.   MultiRowMutation should be delivered to the correct region.
bq.   
bq.   I think we agree on the above.

Oh... MultiRowMutation does not know anything about regions. So we have the use 
the one passed.


- Lars


  was (Author: jirapos...@reviews.apache.org):


bq.  On 2012-02-03 05:07:24, Ted Yu wrote:
bq.   
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java,
 line 3160
bq.   https://reviews.apache.org/r/3748/diff/1/?file=72045#file72045line3160
bq.  
bq.   My point was we shouldn't throw exception in this case.
bq.   MultiRowMutation should be delivered to the correct region.
bq.   
bq.   I think we agree on the above.

Oh... MultiRowMutation does not know anything about regions. So we have the use 
the one passed.


- Lars


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3748/#review4791
---


On 2012-02-03 01:29:58, Lars Hofhansl wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/3748/
bq.  ---
bq.  
bq.  (Updated 2012-02-03 01:29:58)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This builds on HBASE-3584, HBASE-5203, and HBASE-5304.
bq.  
bq.  Multiple Rows can be locked and applied atomically as long as the 
application ensures that all rows reside in the same Region (by presplitting or 
a custom RegionSplitPolicy).
bq.  At SFDC we can use this to colocate subsets of a tenant's data and allow 
atomic operations over these subsets.
bq.  
bq.  Obviously this is an advanced features and this prominently called out in 
the Javadoc.
bq.  
bq.  
bq.  This addresses bug HBASE-5229.
bq.  https://issues.apache.org/jira/browse/HBASE-5229
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiRowMutation.java
 PRE-CREATION 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/RowMutation.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java
 1239953 
bq.  
bq.  Diff: https://reviews.apache.org/r/3748/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  Tests added to TestFromClientSide and TestAtomicOperation
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Lars
bq.  
bq.


  
 Explore building blocks for multi-row local transactions.
 ---

 Key: HBASE-5229
 URL: https://issues.apache.org/jira/browse/HBASE-5229
 Project: HBase
  Issue Type: New Feature
  Components: client, regionserver
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.0

 Attachments: 5229-multiRow.txt,

[jira] [Issue Comment Edited] (HBASE-5229) Explore building blocks for multi-row local transactions.

2012-02-03 Thread Lars Hofhansl (Issue Comment Edited) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200104#comment-13200104
 ] 

Lars Hofhansl edited comment on HBASE-5229 at 2/3/12 10:29 PM:
---


bq.  On 2012-02-03 05:01:34, Jimmy Xiang wrote:
bq.   
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiRowMutation.java,
 line 67
bq.   https://reviews.apache.org/r/3748/diff/1/?file=72039#file72039line67
bq.  
bq.   Should we check they are for different rows here?
bq.  
bq.  Lars Hofhansl wrote:
bq.  It is still valid to pass the same rows here.
bq.  
bq.  Jimmy Xiang wrote:
bq.  That's right.  Should the server side share the same function to do 
MultiRowMutation and RowMutation?  Looks the server side doesn't have much 
difference for mutateRow() and mutateRows()

Oh... I misunderstood what you were saying.
On the server it is mostly the same code. On the client I would like to keep 
them separate features as RowMutation (hopefully) is a pretty standard feature, 
whereas MultiRowMutation is pretty advanced (need to manage your Splits, etc) 
and really treads new HBase ground.


- Lars


  was (Author: jirapos...@reviews.apache.org):


bq.  On 2012-02-03 05:01:34, Jimmy Xiang wrote:
bq.   
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiRowMutation.java,
 line 67
bq.   https://reviews.apache.org/r/3748/diff/1/?file=72039#file72039line67
bq.  
bq.   Should we check they are for different rows here?
bq.  
bq.  Lars Hofhansl wrote:
bq.  It is still valid to pass the same rows here.
bq.  
bq.  Jimmy Xiang wrote:
bq.  That's right.  Should the server side share the same function to do 
MultiRowMutation and RowMutation?  Looks the server side doesn't have much 
difference for mutateRow() and mutateRows()

Oh... I misunderstood what you were saying.
On the server it is mostly the same code. On the client I would like to keep 
them separate features as RowMutation (hopefully) is a pretty standard feature, 
whereas MultiRowMutation is pretty advanced (need to manage your Splits, etc) 
and really treads new HBase ground.


- Lars


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3748/#review4789
---


On 2012-02-03 19:59:55, Lars Hofhansl wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/3748/
bq.  ---
bq.  
bq.  (Updated 2012-02-03 19:59:55)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This builds on HBASE-3584, HBASE-5203, and HBASE-5304.
bq.  
bq.  Multiple Rows can be locked and applied atomically as long as the 
application ensures that all rows reside in the same Region (by presplitting or 
a custom RegionSplitPolicy).
bq.  At SFDC we can use this to colocate subsets of a tenant's data and allow 
atomic operations over these subsets.
bq.  
bq.  Obviously this is an advanced features and this prominently called out in 
the Javadoc.
bq.  
bq.  
bq.  This addresses bug HBASE-5229.
bq.  https://issues.apache.org/jira/browse/HBASE-5229
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiRowMutation.java
 PRE-CREATION 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
 1239953 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java
 1239953 
bq.  
bq.  Diff:

[jira] [Updated] (HBASE-5229) Explore building blocks for multi-row local transactions.

2012-02-03 Thread Lars Hofhansl (Updated) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Lars Hofhansl updated HBASE-5229:
-

Status: Patch Available (was: Reopened)

Explore building blocks for multi-row local transactions.
---

Attachments: 5229-multiRow-v2.txt, 5229-multiRow.txt,
5229-seekto-v2.txt, 5229-seekto.txt, 5229.txt

[jira] [Updated] (HBASE-5229) Explore building blocks for multi-row local transactions.

2012-02-03 Thread Lars Hofhansl (Updated) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Lars Hofhansl updated HBASE-5229:
-

Attachment: 5229-multiRow-v2.txt

@Ted... Getting test run in.

Explore building blocks for multi-row local transactions.
---

Attachments: 5229-multiRow-v2.txt, 5229-multiRow.txt,
5229-seekto-v2.txt, 5229-seekto.txt, 5229.txt

[jira] [Commented] (HBASE-5318) Support Eclipse Indigo

[
https://issues.apache.org/jira/browse/HBASE-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200145#comment-13200145
]

Hadoop QA commented on HBASE-5318:
--

-1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12513187/mvn_HBASE-5318_r1.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.

-1 javadoc. The javadoc tool appears to have generated -136 warning
messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 156 new Findbugs (version
1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestAdmin

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/901//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/901//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/901//console

This message is automatically generated.

Support Eclipse Indigo
---

[jira] [Commented] (HBASE-5318) Support Eclipse Indigo

[
https://issues.apache.org/jira/browse/HBASE-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200162#comment-13200162
]

Jesse Yates commented on HBASE-5318:

TestAdmin passes locally. I think this is good to go.

Support Eclipse Indigo
---

[jira] [Commented] (HBASE-5229) Explore building blocks for multi-row local transactions.

[
https://issues.apache.org/jira/browse/HBASE-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200164#comment-13200164
]

Hadoop QA commented on HBASE-5229:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12513190/5229-multiRow-v2.txt
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 6 new or modified tests.

-1 javadoc. The javadoc tool appears to have generated -136 warning
messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 156 new Findbugs (version
1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.io.hfile.TestHFileBlock
org.apache.hadoop.hbase.mapreduce.TestImportTsv

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/902//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/902//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/902//console

This message is automatically generated.

Explore building blocks for multi-row local transactions.
---

Attachments: 5229-multiRow-v2.txt, 5229-multiRow.txt,
5229-seekto-v2.txt, 5229-seekto.txt, 5229.txt

[jira] [Created] (HBASE-5332) Deterministic Compaction Jitter

2012-02-03 Thread Nicolas Spiegelberg (Created) (JIRA)

Deterministic Compaction Jitter
---

 Key: HBASE-5332
 URL: https://issues.apache.org/jira/browse/HBASE-5332
 Project: HBase
  Issue Type: Improvement
Reporter: Nicolas Spiegelberg
Priority: Minor


Currently, we add jitter to a compaction using delay + jitter*(1 - 
2*Math.random()).  Since this is non-deterministic, we can get major 
compaction storms on server restart as half the Stores that were set to delay 
+ jitter will now be set to delay - jitter.  We need a more deterministic 
way to jitter major compactions so this information can persist across server 
restarts.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-5332) Deterministic Compaction Jitter

2012-02-03 Thread Nicolas Spiegelberg (Assigned) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicolas Spiegelberg reassigned HBASE-5332:
--

Assignee: Nicolas Spiegelberg

 Deterministic Compaction Jitter
 ---

 Key: HBASE-5332
 URL: https://issues.apache.org/jira/browse/HBASE-5332
 Project: HBase
  Issue Type: Improvement
Reporter: Nicolas Spiegelberg
Assignee: Nicolas Spiegelberg
Priority: Minor

 Currently, we add jitter to a compaction using delay + jitter*(1 - 
 2*Math.random()).  Since this is non-deterministic, we can get major 
 compaction storms on server restart as half the Stores that were set to 
 delay + jitter will now be set to delay - jitter.  We need a more 
 deterministic way to jitter major compactions so this information can persist 
 across server restarts.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5318) Support Eclipse Indigo

2012-02-03 Thread Nicolas Spiegelberg (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200221#comment-13200221
]

Zhihong Yu commented on HBASE-5318:
---

That's right.

Support Eclipse Indigo
---

[jira] [Created] (HBASE-5333) Introduce Memstore backpressure for writes

2012-02-03 Thread Lars Hofhansl (Created) (JIRA)

Introduce Memstore backpressure for writes


 Key: HBASE-5333
 URL: https://issues.apache.org/jira/browse/HBASE-5333
 Project: HBase
  Issue Type: Improvement
Reporter: Lars Hofhansl


Currently if the memstore/flush/compaction cannot keep up with the writeload, 
we block writers up to hbase.hstore.blockingWaitTime milliseconds (default is 
9).
Would be nice if there was a concept of a soft backpressure that slows 
writing clients gracefully *before* we reach this condition.

From the log:
2012-02-04 00:00:06,963 WARN 
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Region 
table,,1328313512779.c2761757621ddf8fb78baf5288d71271. has too many store 
files; delaying flush up to 9ms


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5332) Deterministic Compaction Jitter


[ 
https://issues.apache.org/jira/browse/HBASE-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200233#comment-13200233
 ] 

Nicolas Spiegelberg commented on HBASE-5332:


The current proposal is to seed the random number generator with existing 
persistent data:
{code}
 java.util.Random(hash(Region Name + oldest file timestamp)).nextDouble()  
{code}

Note that that the currently algorithm is normally an annoyance in 92 and above 
because major compactions will normally go in the large compaction queue.  
However, 90 and users who have the large compaction queue incorrectly 
configured could experience high latency after restart because a lot of major 
compactions will pile up.

 Deterministic Compaction Jitter
 ---

 Key: HBASE-5332
 URL: https://issues.apache.org/jira/browse/HBASE-5332
 Project: HBase
  Issue Type: Improvement
Reporter: Nicolas Spiegelberg
Assignee: Nicolas Spiegelberg
Priority: Minor

 Currently, we add jitter to a compaction using delay + jitter*(1 - 
 2*Math.random()).  Since this is non-deterministic, we can get major 
 compaction storms on server restart as half the Stores that were set to 
 delay + jitter will now be set to delay - jitter.  We need a more 
 deterministic way to jitter major compactions so this information can persist 
 across server restarts.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5229) Explore building blocks for multi-row local transactions.

2012-02-03 Thread Nicolas Spiegelberg (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200236#comment-13200236
]

Lars Hofhansl commented on HBASE-5229:
--

Test run looks good.

Explore building blocks for multi-row local transactions.
---

Attachments: 5229-multiRow-v2.txt, 5229-multiRow.txt,
5229-seekto-v2.txt, 5229-seekto.txt, 5229.txt

[jira] [Created] (HBASE-5334) Pluggable Compaction Algorithms

2012-02-03 Thread Nicolas Spiegelberg (Created) (JIRA)

Pluggable Compaction Algorithms
---

 Key: HBASE-5334
 URL: https://issues.apache.org/jira/browse/HBASE-5334
 Project: HBase
  Issue Type: Improvement
Reporter: Nicolas Spiegelberg
Assignee: Nicolas Spiegelberg
Priority: Minor


It would be good to create a set of common compaction algorithms so that we can 
tune this on a per-CF basis.  In order to accomplish this, we need to refactor 
the current algorithm for plugability.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5335) Dynamic Schema Configurations

2012-02-03 Thread Nicolas Spiegelberg (Created) (JIRA)

Dynamic Schema Configurations
-

 Key: HBASE-5335
 URL: https://issues.apache.org/jira/browse/HBASE-5335
 Project: HBase
  Issue Type: New Feature
Reporter: Nicolas Spiegelberg
Assignee: Nicolas Spiegelberg


Currently, the ability for a core developer to add per-table  per-CF 
configuration settings is very heavyweight.  You need to add a reserved keyword 
all the way up the stack  you have to support this variable long-term if 
you're going to expose it explicitly to the user.  This has ended up with using 
Configuration.get() a lot because it is lightweight and you can tweak settings 
while you're trying to understand system behavior [since there are many config 
params that may never need to be tuned].  We need to add the ability to put  
read arbitrary KV settings in the HBase schema.  Combined with online schema 
change, this will allow us to safely iterate on configuration settings.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5335) Dynamic Schema Configurations

[
https://issues.apache.org/jira/browse/HBASE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200269#comment-13200269
]

Nicolas Spiegelberg commented on HBASE-5335:

My current proposal is to create a derived class off HBaseConfiguration called
HBaseTableConfiguration. Also, create a HBaseStoreConfiguration derived from
HBaseTableConfiguration. This will allow us to specify all the normal
Config.get() keys on a per-table, per-CF basis without a major code refactor.
Still need to flush out how this would look in the schema, since you probably
want to distinguish between reserved keywords and at your own risk hidden
settings.

For config items that we later identify as important, we can create a reserved
keyword and then map it to the old config name. This will allow us to iterate
fast stabilize without needing explicit schema migration.

Dynamic Schema Configurations
-

Key: HBASE-5335
URL: https://issues.apache.org/jira/browse/HBASE-5335
Project: HBase
Issue Type: New Feature
Reporter: Nicolas Spiegelberg
Assignee: Nicolas Spiegelberg
Labels: configuration, schema

Currently, the ability for a core developer to add per-table per-CF
configuration settings is very heavyweight. You need to add a reserved
keyword all the way up the stack you have to support this variable
long-term if you're going to expose it explicitly to the user. This has
ended up with using Configuration.get() a lot because it is lightweight and
you can tweak settings while you're trying to understand system behavior
[since there are many config params that may never need to be tuned]. We
need to add the ability to put read arbitrary KV settings in the HBase
schema. Combined with online schema change, this will allow us to safely
iterate on configuration settings.

[jira] [Created] (HBASE-5336) Spurious exceptions in HConnectionImplementation

2012-02-03 Thread Lars Hofhansl (Created) (JIRA)

Spurious exceptions in HConnectionImplementation


 Key: HBASE-5336
 URL: https://issues.apache.org/jira/browse/HBASE-5336
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl


I have seen this on the client a few time during heave write testing:

java.util.concurrent.ExecutionException: java.io.IOException: 
java.io.IOException: java.lang.NullPointerException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1524)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1376)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:891)
at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:743)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:730)
at NewsFeedCreate.insert(NewsFeedCreate.java:91)
at NewsFeedCreate$1.run(NewsFeedCreate.java:38)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: java.io.IOException: 
java.lang.NullPointerException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at 
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
at 
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
at 
org.apache.hadoop.hbase.client.ServerCallable.translateException(ServerCallable.java:228)
at 
org.apache.hadoop.hbase.client.ServerCallable.withoutRetries(ServerCallable.java:212)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1360)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1348)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
... 1 more
Caused by: org.apache.hadoop.ipc.RemoteException: java.io.IOException: 
java.lang.NullPointerException
at 
org.apache.hadoop.io.SequenceFile$Writer.getLength(SequenceFile.java:1099)
at 
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.getLength(SequenceFileLogWriter.java:243)
at org.apache.hadoop.hbase.regionserver.wal.HLog.syncer(HLog.java:1289)
at org.apache.hadoop.hbase.regionserver.wal.HLog.sync(HLog.java:1386)
at 
org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchPut(HRegion.java:2161)
at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:1954)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3363)
at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at 
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
at 
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)

at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:899)
at 
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
at $Proxy1.multi(Unknown Source)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1353)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1351)
at 
org.apache.hadoop.hbase.client.ServerCallable.withoutRetries(ServerCallable.java:210)
... 7 more


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5317) Fix TestHFileOutputFormat to work against hadoop 0.23

2012-02-03 Thread Gregory Chanan (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gregory Chanan updated HBASE-5317:
--

Attachment: HBASE-5317-v0.patch

Patch v0 fixes test_TIMERANGE and testColumnFamilyCompression

Other tests are a little more involved.

 Fix TestHFileOutputFormat to work against hadoop 0.23
 -

 Key: HBASE-5317
 URL: https://issues.apache.org/jira/browse/HBASE-5317
 Project: HBase
  Issue Type: Bug
  Components: test
Affects Versions: 0.94.0, 0.92.0
Reporter: Gregory Chanan
Assignee: Gregory Chanan
 Attachments: HBASE-5317-v0.patch


 Running
 mvn -Dhadoop.profile=23 test -P localTests 
 -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
 yields this on 0.92:
 Failed tests:   
 testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
  HFile for column family info-A not found
 Tests in error: 
   test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): 
 /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_00_0
  (Is a directory)
   
 testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
  TestTable
   
 testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
  TestTable
 It looks like on trunk, this also results in an error:
   
 testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
  TestTable
 I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but 
 haven't fixed the other 3 yet.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5318) Support Eclipse Indigo

2012-02-03 Thread Nicolas Spiegelberg (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200301#comment-13200301
]

Zhihong Yu commented on HBASE-5318:
---

Integrated to TRUNK.

Thanks for the patch, Jesse.

Thanks for the review, Lars

Support Eclipse Indigo
---

[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work against hadoop 0.23

2012-02-03 Thread Gregory Chanan (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200303#comment-13200303
 ] 

Gregory Chanan commented on HBASE-5317:
---

testMRIncrementalLoad (and probably the other 2) fail with TestTable; this is 
pretty easy to get past.

But after that MR Job fails with:
2012-02-03 17:02:37,982 FATAL [main] 
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
org.apache.hadoop.yarn.YarnException: java.lang.RuntimeException: 
java.lang.ClassNotFoundException: 
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat
at 
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.createOutputCommitter(MRAppMaster.java:320)
at 
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.init(MRAppMaster.java:213)

 Fix TestHFileOutputFormat to work against hadoop 0.23
 -

 Key: HBASE-5317
 URL: https://issues.apache.org/jira/browse/HBASE-5317
 Project: HBase
  Issue Type: Bug
  Components: test
Affects Versions: 0.94.0, 0.92.0
Reporter: Gregory Chanan
Assignee: Gregory Chanan
 Attachments: HBASE-5317-v0.patch


 Running
 mvn -Dhadoop.profile=23 test -P localTests 
 -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
 yields this on 0.92:
 Failed tests:   
 testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
  HFile for column family info-A not found
 Tests in error: 
   test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): 
 /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_00_0
  (Is a directory)
   
 testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
  TestTable
   
 testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
  TestTable
 It looks like on trunk, this also results in an error:
   
 testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
  TestTable
 I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but 
 haven't fixed the other 3 yet.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5330) TestCompactSelection - adding 2 test cases to testCompactionRatio


[ 
https://issues.apache.org/jira/browse/HBASE-5330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200311#comment-13200311
 ] 

Nicolas Spiegelberg commented on HBASE-5330:


There's actually more subtle shortcomings that these tests expose.  

1) The major compaction is treated differently than the reference use case 
because a major compaction does not require all the files.  A major compaction 
just requires a contiguous range of the oldest files.  We need to change 
compactSelection to return whether the operation is a major compaction or not.

2) The reference use case should not look all all the StoreFiles.  Instead, it 
should compact range(oldest_reference, newest_reference).

3) I don't understand why reference files are [5:1].  It should be [7:3] like 
the others.

 TestCompactSelection - adding 2 test cases to testCompactionRatio
 -

 Key: HBASE-5330
 URL: https://issues.apache.org/jira/browse/HBASE-5330
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: TestCompactSelection_hbase_5330.java.patch


 There were three existing assertions in TestCompactSelection 
 testCompactionRatio that did max # of files assertions...
 {code}
 assertEquals(maxFiles,
 
 store.compactSelection(sfCreate(7,6,5,4,3,2,1)).getFilesToCompact().size());
 {code}
 ... and for references ...
 {code}
   assertEquals(maxFiles,
 store.compactSelection(sfCreate(true, 
 7,6,5,4,3,2,1)).getFilesToCompact().size());
 {code}
  
 ... but they didn't assert against which StoreFiles got selected.  While the 
 number of StoreFiles is the same, the files selected are actually different, 
 and I thought that there should be explicit assertions showing that.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5318) Support Eclipse Indigo

2012-02-03 Thread ramkrishna.s.vasudevan (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200324#comment-13200324
 ] 

Hudson commented on HBASE-5318:
---

Integrated in HBase-TRUNK #2654 (See 
[https://builds.apache.org/job/HBase-TRUNK/2654/])
HBASE-5318 Support Eclipse Indigo (Jesse Yates)
REVERT HBASE-5318 Support Eclipse Indigo

tedyu : 
Files : 
* /hbase/trunk/pom.xml

larsh : 
Files : 
* /hbase/trunk/pom.xml


 Support Eclipse Indigo 
 ---

 Key: HBASE-5318
 URL: https://issues.apache.org/jira/browse/HBASE-5318
 Project: HBase
  Issue Type: Improvement
  Components: build
Affects Versions: 0.94.0
 Environment: Eclipse Indigo (1.4.1) which includes m2eclipse (1.0 
 SR1).
Reporter: Jesse Yates
Assignee: Jesse Yates
Priority: Minor
  Labels: maven
 Attachments: mvn_HBASE-5318_r0.patch, mvn_HBASE-5318_r1.patch


 The current 'standard' release of Eclipse (indigo) comes with m2eclipse 
 installed. However, as of m2e v1.0, interesting lifecycle phases are now 
 handled via a 'connector'. However, several of the plugins we use don't 
 support connectors. This means that eclipse bails out and won't build the 
 project or view it as 'working' even though it builds just fine from the the 
 command line.
 Since Eclipse is one of the major java IDEs and that Indigo has been around 
 for a while, we should make it easy to for new devs to pick up the code and 
 for older devs to upgrade painlessly. The original build should not be 
 modified in any significant way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5323) Need to handle assertion error while splitting log through ServerShutDownHandler by shutting down the master


[ 
https://issues.apache.org/jira/browse/HBASE-5323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200329#comment-13200329
 ] 

ramkrishna.s.vasudevan commented on HBASE-5323:
---

@Todd

Ok.. I can update the patch accordingly. I agree with your suggestion.

 Need to handle assertion error while splitting log through 
 ServerShutDownHandler by shutting down the master
 

 Key: HBASE-5323
 URL: https://issues.apache.org/jira/browse/HBASE-5323
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.5
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.94.0, 0.90.7

 Attachments: HBASE-5323.patch


 We know that while parsing the HLog we expect the proper length from HDFS.
 In WALReaderFSDataInputStream
 {code}
   assert(realLength = this.length);
 {code}
 We are trying to come out if the above condition is not satisfied.  But if 
 SSH.splitLog() gets this problem then it lands in the run method of 
 EventHandler.  This kills the SSH thread and so further assignment does not 
 happen.  If ROOT and META are to be assigned they cannot be.
 I think in this condition we abort the master by catching such exceptions.
 Please do suggest.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5335) Dynamic Schema Configurations


[ 
https://issues.apache.org/jira/browse/HBASE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200330#comment-13200330
 ] 

Lars Hofhansl commented on HBASE-5335:
--

We were just discussing here at SFDC to add a generalized way to serialized 
Configurations into a HTableDescriptor. What you propose is much cleaner than 
what we had in mind. So +1 :)

Are you planning some kind of classification theme to distinguish config 
settings that only make sense globally from those that make sense per CF/Table?

Short term I was planning to add the ability to set arbitrary HTableDescriptor 
value through the shell, I'll hold of on that.

 Dynamic Schema Configurations
 -

 Key: HBASE-5335
 URL: https://issues.apache.org/jira/browse/HBASE-5335
 Project: HBase
  Issue Type: New Feature
Reporter: Nicolas Spiegelberg
Assignee: Nicolas Spiegelberg
  Labels: configuration, schema

 Currently, the ability for a core developer to add per-table  per-CF 
 configuration settings is very heavyweight.  You need to add a reserved 
 keyword all the way up the stack  you have to support this variable 
 long-term if you're going to expose it explicitly to the user.  This has 
 ended up with using Configuration.get() a lot because it is lightweight and 
 you can tweak settings while you're trying to understand system behavior 
 [since there are many config params that may never need to be tuned].  We 
 need to add the ability to put  read arbitrary KV settings in the HBase 
 schema.  Combined with online schema change, this will allow us to safely 
 iterate on configuration settings.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5323) Need to handle assertion error while splitting log through ServerShutDownHandler by shutting down the master

[
https://issues.apache.org/jira/browse/HBASE-5323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200331#comment-13200331
]

Lars Hofhansl commented on HBASE-5323:
--

+1 on aborting the Master when this happens
+1 on changing the assert to try/catch.

Generally assert should only be used with conditions that we can live with in a
production environment (as Todd points out).

Need to handle assertion error while splitting log through
ServerShutDownHandler by shutting down the master

Attachments: HBASE-5323.patch

[jira] [Updated] (HBASE-5318) Support Eclipse Indigo

[
https://issues.apache.org/jira/browse/HBASE-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Zhihong Yu updated HBASE-5318:
--

Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)

Support Eclipse Indigo
---

[jira] [Commented] (HBASE-5329) addRowLock() may allocate duplicate lock id, causing the client to be blocked

2012-02-03 Thread ramkrishna.s.vasudevan (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200332#comment-13200332
 ] 

Lars Hofhansl commented on HBASE-5329:
--

I had raised that fact about getScanner() a while ago... Isn't that 
*incredibly* unlikely, though.
nextLong generates all 2^64 possible longs with roughly equal probability.
The chances of the same long being generated are astronomically small.

From the log attached, you actually saw this happening? Are you sure this due 
to a duplicate random number?


 addRowLock() may allocate duplicate lock id, causing the client to be blocked
 -

 Key: HBASE-5329
 URL: https://issues.apache.org/jira/browse/HBASE-5329
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.3
 Environment: Red Hat Enterprise Linux Server release 5.4 
Reporter: liaoxiangui
Assignee: Zhihong Yu
Priority: Critical

 {code}
 protected long addRowLock(Integer r, HRegion region) throws 
 LeaseStillHeldException
 {
   long lockId = -1L;
   lockId = rand.nextLong();   //!!!may generate duplicated 
 id，bug?
   String lockName = String.valueOf(lockId);
   rowlocks.put(lockName, r);
   this.leases.createLease(lockName, new RowLockListener(lockName, 
 region));
   return lockId;
 }
 {code}
 In addRowLock(),rand may generate duplicated lock id, it may cause 
 regionserver throw exception(Leases$LeaseStillHeldException).The client will 
 be blocked until old rowlock is released.
 {code}
 2012-02-03 15:21:50,084 ERROR 
 org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock 
 (fsOk: true)
 org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException
 at 
 org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:1986)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:1963)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5200) AM.ProcessRegionInTransition() and AM.handleRegion() race thus leaving the region assignment inconsistent


[ 
https://issues.apache.org/jira/browse/HBASE-5200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200334#comment-13200334
 ] 

ramkrishna.s.vasudevan commented on HBASE-5200:
---

+1 on v2 patch. 

 AM.ProcessRegionInTransition() and AM.handleRegion() race thus leaving the 
 region assignment inconsistent
 -

 Key: HBASE-5200
 URL: https://issues.apache.org/jira/browse/HBASE-5200
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.5
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.94.0, 0.90.7, 0.92.1

 Attachments: 5200-v2.txt, HBASE-5200.patch, HBASE-5200_1.patch, 
 TEST-org.apache.hadoop.hbase.master.TestRestartCluster.xml


 This is the scenario
 Consider a case where the balancer is going on thus trying to close regions 
 in a RS.
 Before we could close a master switch happens.  
 On Master switch the set of nodes that are in RIT is collected and we first 
 get Data and start watching the node
 After that the node data is added into RIT.
 Now by this time (before adding to RIT) if the RS to which close was called 
 does a transition in AM.handleRegion() we miss the handling saying RIT state 
 was null.
 {code}
 2012-01-13 10:50:46,358 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 a66d281d231dfcaea97c270698b26b6f from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,358 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 c12e53bfd48ddc5eec507d66821c4d23 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,358 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 59ae13de8c1eb325a0dd51f4902d2052 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 f45bc9614d7575f35244849af85aa078 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 cc3ecd7054fe6cd4a1159ed92fd62641 from server 
 HOST-192-168-47-204,20020,1326342744518 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 3af40478a17fee96b4a192b22c90d5a2 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 e6096a8466e730463e10d3d61f809b92 from server 
 HOST-192-168-47-204,20020,1326342744518 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 4806781a1a23066f7baed22b4d237e24 from server 
 HOST-192-168-47-204,20020,1326342744518 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 2012-01-13 10:50:46,359 WARN 
 org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region 
 d69e104131accaefe21dcc01fddc7629 from server 
 HOST-192-168-47-205,20020,1326363111288 but region was in  the state null and 
 not in expected PENDING_CLOSE or CLOSING states
 {code}
 In branch the CLOSING node is created by RS thus leading to more 
 inconsistency.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5329) addRowLock() may allocate duplicate lock id, causing the client to be blocked


[ 
https://issues.apache.org/jira/browse/HBASE-5329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200337#comment-13200337
 ] 

Zhihong Yu commented on HBASE-5329:
---

Although rare, this is possible - considering duplicate scanner Id with lock 
Id, etc.
Here is related code:
{code}
  if (leases.containsKey(lease.getLeaseName())) {
throw new LeaseStillHeldException(lease.getLeaseName());
  }
{code}

 addRowLock() may allocate duplicate lock id, causing the client to be blocked
 -

 Key: HBASE-5329
 URL: https://issues.apache.org/jira/browse/HBASE-5329
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.90.3
 Environment: Red Hat Enterprise Linux Server release 5.4 
Reporter: liaoxiangui
Assignee: Zhihong Yu
Priority: Critical

 {code}
 protected long addRowLock(Integer r, HRegion region) throws 
 LeaseStillHeldException
 {
   long lockId = -1L;
   lockId = rand.nextLong();   //!!!may generate duplicated 
 id，bug?
   String lockName = String.valueOf(lockId);
   rowlocks.put(lockName, r);
   this.leases.createLease(lockName, new RowLockListener(lockName, 
 region));
   return lockId;
 }
 {code}
 In addRowLock(),rand may generate duplicated lock id, it may cause 
 regionserver throw exception(Leases$LeaseStillHeldException).The client will 
 be blocked until old rowlock is released.
 {code}
 2012-02-03 15:21:50,084 ERROR 
 org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock 
 (fsOk: true)
 org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException
 at 
 org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:1986)
 at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:1963)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4658) Put attributes are not exposed via the ThriftServer


[ 
https://issues.apache.org/jira/browse/HBASE-4658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200340#comment-13200340
 ] 

Hudson commented on HBASE-4658:
---

Integrated in HBase-TRUNK-security #100 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/100/])
HBASE-4658 Put attributes are not exposed via the ThriftServer (Dhruba)

tedyu : 
Files : 
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/thrift/generated/Hbase.java
* /hbase/trunk/src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java


 Put attributes are not exposed via the ThriftServer
 ---

 Key: HBASE-4658
 URL: https://issues.apache.org/jira/browse/HBASE-4658
 Project: HBase
  Issue Type: Bug
  Components: thrift
Reporter: dhruba borthakur
Assignee: dhruba borthakur
 Attachments: D1563.1.patch, D1563.1.patch, D1563.1.patch, 
 D1563.2.patch, D1563.2.patch, D1563.2.patch, D1563.3.patch, D1563.3.patch, 
 D1563.3.patch, ThriftPutAttributes1.txt


 The Put api also takes in a bunch of arbitrary attributes that an application 
 can use to associate metadata with each put operation. This is not exposed 
 via Thrift.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5318) Support Eclipse Indigo


[ 
https://issues.apache.org/jira/browse/HBASE-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200341#comment-13200341
 ] 

Hudson commented on HBASE-5318:
---

Integrated in HBase-TRUNK-security #100 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/100/])
HBASE-5318 Support Eclipse Indigo (Jesse Yates)
REVERT HBASE-5318 Support Eclipse Indigo

tedyu : 
Files : 
* /hbase/trunk/pom.xml

larsh : 
Files : 
* /hbase/trunk/pom.xml


 Support Eclipse Indigo 
 ---

 Key: HBASE-5318
 URL: https://issues.apache.org/jira/browse/HBASE-5318
 Project: HBase
  Issue Type: Improvement
  Components: build
Affects Versions: 0.94.0
 Environment: Eclipse Indigo (1.4.1) which includes m2eclipse (1.0 
 SR1).
Reporter: Jesse Yates
Assignee: Jesse Yates
Priority: Minor
  Labels: maven
 Attachments: mvn_HBASE-5318_r0.patch, mvn_HBASE-5318_r1.patch


 The current 'standard' release of Eclipse (indigo) comes with m2eclipse 
 installed. However, as of m2e v1.0, interesting lifecycle phases are now 
 handled via a 'connector'. However, several of the plugins we use don't 
 support connectors. This means that eclipse bails out and won't build the 
 project or view it as 'working' even though it builds just fine from the the 
 command line.
 Since Eclipse is one of the major java IDEs and that Indigo has been around 
 for a while, we should make it easy to for new devs to pick up the code and 
 for older devs to upgrade painlessly. The original build should not be 
 modified in any significant way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work against hadoop 0.23