date:20120303

[
https://issues.apache.org/jira/browse/HBASE-5510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221547#comment-13221547
]

Zhihong Yu commented on HBASE-5510:
---

Can I see the patch ?

Change in LB.randomAssignment(ListServerName servers) API
---

Key: HBASE-5510
URL: https://issues.apache.org/jira/browse/HBASE-5510
Project: HBase
Issue Type: Improvement
Affects Versions: 0.92.0
Reporter: Anoop Sam John
Assignee: ramkrishna.s.vasudevan

In LB there is randomAssignment(ListServerName servers) API which will be
used by AM to assign
a region from a down RS. [This will be also used in other cases like call to
assign() API from client]
I feel it would be better to pass the HRegionInfo also into this method.
When the LB making a choice for a region
assignment, when one RS is down, it would be nice that the LB knows for
which region it is doing this server selection.
+Scenario+
While one RS down, we wanted the regions to get moved to other RSs but a set
of regions stay together. We are having custom load balancer but with the
current way of LB interface this is not possible. Another way is I can allow
a random assignment of the regions at the RS down time. Later with a cluster
balance I can balance the regions as I need. But this might make regions
assign 1st to one RS and then again move to another. Also for some time
period my business use case can not get satisfied.
Also I have seen some issue in JIRA which speaks about making sure that Root
and META regions always sit in some specific RSs. With the current LB API
this wont be possible in future.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5510) Change in LB.randomAssignment(ListServerName servers) API

2012-03-03 Thread Anoop Sam John (Updated) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Anoop Sam John updated HBASE-5510:
--

Attachment: HBase-5510.patch

Change in LB.randomAssignment(ListServerName servers) API
---

Key: HBASE-5510
URL: https://issues.apache.org/jira/browse/HBASE-5510
Project: HBase
Issue Type: Improvement
Affects Versions: 0.92.0
Reporter: Anoop Sam John
Assignee: ramkrishna.s.vasudevan
Attachments: HBase-5510.patch

[jira] [Commented] (HBASE-5510) Change in LB.randomAssignment(ListServerName servers) API

2012-03-03 Thread Anoop Sam John (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221556#comment-13221556
]

Anoop Sam John commented on HBASE-5510:
---

Pls see the patch.

Change in LB.randomAssignment(ListServerName servers) API
---

[jira] [Commented] (HBASE-5512) Add support for INCLUDE_AND_SEEK_USING_HINT

2012-03-03 Thread Anoop Sam John (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221559#comment-13221559
 ] 

Anoop Sam John commented on HBASE-5512:
---

ScanQueryMatcher
} else if (filterResponse == ReturnCode.INCLUDE_AND_SEEK_NEXT_USING_HINT) {
+
   }
You need make filterSeek=true? 

 Add support for INCLUDE_AND_SEEK_USING_HINT
 ---

 Key: HBASE-5512
 URL: https://issues.apache.org/jira/browse/HBASE-5512
 Project: HBase
  Issue Type: Improvement
Reporter: Zhihong Yu
Assignee: Lars Hofhansl
 Attachments: 5512.txt


 This came up from HBASE-2038
 From Anoop:
 - What we wanted from the filter is include a row and then seek to the next 
 row which we are interested in. I cant see such a facility with our Filter 
 right now. Correct me if I am wrong. So suppose we already seeked to one row 
 and this need to be included in the result, then the Filter should return 
 INCLUDE. Then when the next next() call happens, then only we can return a 
 SEEK_USING_HINT. So one extra row reading is needed. This might create even 
 one unwanted HFileBlock fetch (who knows).
 Can we add reseek() at higher level?
 From Lars:
 Yep, for that we'd need to add INCLUDE_AND_SEEK_USING_HINT (similar to the 
 INCLUDE_AND_SEEK_NEXT_ROW that we already have). Shouldn't be hard to add, 
 I'm happy to do that, if that's the route we want to go with this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5510) Change in LB.randomAssignment(ListServerName servers) API

2012-03-03 Thread Zhihong Yu (Issue Comment Edited) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221600#comment-13221600
]

Zhihong Yu commented on HBASE-5510:
---

Looking at other methods in LoadBalancer, the regions are passed as first
parameter.
I think randomAssignment() should follow this convention.

Please wrap long lines in the patch.

Currently LoadBalancer is marked @InterfaceAudience.Private
I think the use case presented in this JIRA clearly means it shouldn't be
private.
Please change it to @InterfaceAudience.Public in the next patch.

Change in LB.randomAssignment(ListServerName servers) API
---

[jira] [Issue Comment Edited] (HBASE-5512) Add support for INCLUDE_AND_SEEK_USING_HINT


[ 
https://issues.apache.org/jira/browse/HBASE-5512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221559#comment-13221559
 ] 

Zhihong Yu edited comment on HBASE-5512 at 3/3/12 3:24 PM:
---

ScanQueryMatcher:
{code}
} else if (filterResponse == ReturnCode.INCLUDE_AND_SEEK_NEXT_USING_HINT) {
+
   }
{code}
You need make filterSeek=true? 

  was (Author: anoopsamjohn):
ScanQueryMatcher
} else if (filterResponse == ReturnCode.INCLUDE_AND_SEEK_NEXT_USING_HINT) {
+
   }
You need make filterSeek=true? 
  
 Add support for INCLUDE_AND_SEEK_USING_HINT
 ---

 Key: HBASE-5512
 URL: https://issues.apache.org/jira/browse/HBASE-5512
 Project: HBase
  Issue Type: Improvement
Reporter: Zhihong Yu
Assignee: Lars Hofhansl
 Attachments: 5512.txt


 This came up from HBASE-2038
 From Anoop:
 - What we wanted from the filter is include a row and then seek to the next 
 row which we are interested in. I cant see such a facility with our Filter 
 right now. Correct me if I am wrong. So suppose we already seeked to one row 
 and this need to be included in the result, then the Filter should return 
 INCLUDE. Then when the next next() call happens, then only we can return a 
 SEEK_USING_HINT. So one extra row reading is needed. This might create even 
 one unwanted HFileBlock fetch (who knows).
 Can we add reseek() at higher level?
 From Lars:
 Yep, for that we'd need to add INCLUDE_AND_SEEK_USING_HINT (similar to the 
 INCLUDE_AND_SEEK_NEXT_ROW that we already have). Shouldn't be hard to add, 
 I'm happy to do that, if that's the route we want to go with this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5512) Add support for INCLUDE_AND_SEEK_USING_HINT

2012-03-03 Thread ramkrishna.s.vasudevan (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221638#comment-13221638
 ] 

ramkrishna.s.vasudevan commented on HBASE-5512:
---

@Lars
First of all thanks for the patch Lars.
Anoop and myself were planning to work on this.  :)

 Add support for INCLUDE_AND_SEEK_USING_HINT
 ---

 Key: HBASE-5512
 URL: https://issues.apache.org/jira/browse/HBASE-5512
 Project: HBase
  Issue Type: Improvement
Reporter: Zhihong Yu
Assignee: Lars Hofhansl
 Attachments: 5512.txt


 This came up from HBASE-2038
 From Anoop:
 - What we wanted from the filter is include a row and then seek to the next 
 row which we are interested in. I cant see such a facility with our Filter 
 right now. Correct me if I am wrong. So suppose we already seeked to one row 
 and this need to be included in the result, then the Filter should return 
 INCLUDE. Then when the next next() call happens, then only we can return a 
 SEEK_USING_HINT. So one extra row reading is needed. This might create even 
 one unwanted HFileBlock fetch (who knows).
 Can we add reseek() at higher level?
 From Lars:
 Yep, for that we'd need to add INCLUDE_AND_SEEK_USING_HINT (similar to the 
 INCLUDE_AND_SEEK_NEXT_ROW that we already have). Shouldn't be hard to add, 
 I'm happy to do that, if that's the route we want to go with this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2038) Coprocessors: Region level indexing

2012-03-03 Thread ramkrishna.s.vasudevan (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-2038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221640#comment-13221640
 ] 

ramkrishna.s.vasudevan commented on HBASE-2038:
---

Also we need to have a provision to use the nbRows that is passed while 
scanning to be used in coprocessor such that the normal scanner.next() can be 
used in sync with the cached preScannerNext that we do with nbRows. 

 Coprocessors: Region level indexing
 ---

 Key: HBASE-2038
 URL: https://issues.apache.org/jira/browse/HBASE-2038
 Project: HBase
  Issue Type: New Feature
  Components: coprocessors
Reporter: Andrew Purtell
Priority: Minor

 HBASE-2037 is a good candidate to be done as coprocessor. It also serve as a 
 good goalpost for coprocessor environment design -- there should be enough of 
 it so region level indexing can be reimplemented as a coprocessor without any 
 loss of functionality. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-5286) bin/hbase's logic of adding Hadoop jar files to the classpath is fragile when presented with split packaged Hadoop 0.23 installation

2012-03-03 Thread stack (Resolved) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-5286.
--

   Resolution: Fixed
Fix Version/s: 0.94.0
   0.92.1
 Hadoop Flags: Reviewed

I tried it.  Works.  Default codepath unaffected.  Committed to 0.94, 0.92, and 
trunk.  Thanks for the patch Roman.

 bin/hbase's logic of adding Hadoop jar files to the classpath is fragile when 
 presented with split packaged Hadoop 0.23 installation
 

 Key: HBASE-5286
 URL: https://issues.apache.org/jira/browse/HBASE-5286
 Project: HBase
  Issue Type: Bug
  Components: scripts
Affects Versions: 0.92.0
Reporter: Roman Shaposhnik
Assignee: Roman Shaposhnik
 Fix For: 0.92.1, 0.94.0

 Attachments: HBASE-5286.patch.txt


 Here's the bit from bin/hbase that might need TLC now that Hadoop can be 
 spotted in the wild in split-package configuration:
 {noformat}
 #If avail, add Hadoop to the CLASSPATH and to the JAVA_LIBRARY_PATH
 if [ ! -z $HADOOP_HOME ]; then
   HADOOPCPPATH=
   if [ -z $HADOOP_CONF_DIR ]; then
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} ${HADOOP_HOME}/conf)
   else
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} ${HADOOP_CONF_DIR})
   fi
   if [ `echo ${HADOOP_HOME}/hadoop-core*.jar` != 
 ${HADOOP_HOME}/hadoop-core*.jar ] ; then
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} `ls 
 ${HADOOP_HOME}/hadoop-core*.jar | head -1`)
   else
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} `ls 
 ${HADOOP_HOME}/hadoop-common*.jar | head -1`)
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} `ls 
 ${HADOOP_HOME}/hadoop-hdfs*.jar | head -1`)
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} `ls 
 ${HADOOP_HOME}/hadoop-mapred*.jar | head -1`)
   fi
 {noformat}
 There's a couple of issues with the above code:
0. HADOOP_HOME is now deprecated in Hadoop 0.23
1. the list of jar files added to the class-path should be revised
2. we need to figure out a more robust way to get the jar files that are 
 needed to the classpath (things like hadoop-mapred*.jar tend to match 
 src/test jars as well)
 Better yet, it would be useful to look into whether we can transition HBase's 
 bin/hbase onto using bin/hadoop as a launcher script instead of direct JAVA 
 invocations (Pig, Hive, Sqoop and Mahout already do that)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4890) fix possible NPE in HConnectionManager

2012-03-03 Thread stack (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221643#comment-13221643
 ] 

stack commented on HBASE-4890:
--

This NPE is a bit too easy to manufacture.  Should we hold up 0.92.1 till 
fixed?  Can work on it monday?

 fix possible NPE in HConnectionManager
 --

 Key: HBASE-4890
 URL: https://issues.apache.org/jira/browse/HBASE-4890
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jonathan Hsieh
Priority: Blocker
 Fix For: 0.92.1


 I was running YCSB against a 0.92 branch and encountered this error message:
 {code}
 11/11/29 08:47:16 WARN client.HConnectionManager$HConnectionImplementation: 
 Failed all from 
 region=usertable,user3917479014967760871,1322555655231.f78d161e5724495a9723bcd972f97f41.,
  hostname=c0316.hal.cloudera.com, port=57020
 java.util.concurrent.ExecutionException: java.lang.RuntimeException: 
 java.lang.NullPointerException
 at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
 at java.util.concurrent.FutureTask.get(FutureTask.java:83)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1501)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1353)
 at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:898)
 at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:775)
 at org.apache.hadoop.hbase.client.HTable.put(HTable.java:750)
 at com.yahoo.ycsb.db.HBaseClient.update(Unknown Source)
 at com.yahoo.ycsb.DBWrapper.update(Unknown Source)
 at com.yahoo.ycsb.workloads.CoreWorkload.doTransactionUpdate(Unknown 
 Source)
 at com.yahoo.ycsb.workloads.CoreWorkload.doTransaction(Unknown Source)
 at com.yahoo.ycsb.ClientThread.run(Unknown Source)
 Caused by: java.lang.RuntimeException: java.lang.NullPointerException
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithoutRetries(HConnectionManager.java:1315)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1327)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1325)
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:619)
 Caused by: java.lang.NullPointerException
 at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:158)
 at $Proxy4.multi(Unknown Source)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1330)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1328)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithoutRetries(HConnectionManager.java:1309)
 ... 7 more
 {code}
 It looks like the NPE is caused by server being null in the MultiRespone 
 call() method.
 {code}
  public MultiResponse call() throws IOException {
  return getRegionServerWithoutRetries(
  new ServerCallableMultiResponse(connection, tableName, null) {
public MultiResponse call() throws IOException {
  return server.multi(multi);
}
@Override
public void connect(boolean reload) throws IOException {
  server =
connection.getHRegionConnection(loc.getHostname(), 
 loc.getPort());
}
  }
  );
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5516) GZip leading to memory leak in 0.90. Fix similar to HBASE-5387 needed for 0.90.

2012-03-03 Thread ramkrishna.s.vasudevan (Created) (JIRA)

GZip leading to memory leak in 0.90.  Fix similar to HBASE-5387 needed for 0.90.


 Key: HBASE-5516
 URL: https://issues.apache.org/jira/browse/HBASE-5516
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.5
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.90.7


Usage of GZip is leading to resident memory leak in 0.90.
We need to have something similar to HBASE-5387 in 0.90. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5516) GZip leading to memory leak in 0.90. Fix similar to HBASE-5387 needed for 0.90.

2012-03-03 Thread ramkrishna.s.vasudevan (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221647#comment-13221647
 ] 

ramkrishna.s.vasudevan commented on HBASE-5516:
---

We have an internal patch.  But it requires some more formatting while writing 
the header.  Will upload once done and tested.

 GZip leading to memory leak in 0.90.  Fix similar to HBASE-5387 needed for 
 0.90.
 

 Key: HBASE-5516
 URL: https://issues.apache.org/jira/browse/HBASE-5516
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.5
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.90.7


 Usage of GZip is leading to resident memory leak in 0.90.
 We need to have something similar to HBASE-5387 in 0.90. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5286) bin/hbase's logic of adding Hadoop jar files to the classpath is fragile when presented with split packaged Hadoop 0.23 installation


[ 
https://issues.apache.org/jira/browse/HBASE-5286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221652#comment-13221652
 ] 

Hudson commented on HBASE-5286:
---

Integrated in HBase-0.94 #11 (See 
[https://builds.apache.org/job/HBase-0.94/11/])
HBASE-5286 bin/hbase's logic of adding Hadoop jar files to the classpath is 
fragile when presented with split packaged Hadoop 0.23 installation (Revision 
1296664)

 Result = SUCCESS
stack : 
Files : 
* /hbase/branches/0.94/bin/hbase
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/GetJavaProperty.java


 bin/hbase's logic of adding Hadoop jar files to the classpath is fragile when 
 presented with split packaged Hadoop 0.23 installation
 

 Key: HBASE-5286
 URL: https://issues.apache.org/jira/browse/HBASE-5286
 Project: HBase
  Issue Type: Bug
  Components: scripts
Affects Versions: 0.92.0
Reporter: Roman Shaposhnik
Assignee: Roman Shaposhnik
 Fix For: 0.92.1, 0.94.0

 Attachments: HBASE-5286.patch.txt


 Here's the bit from bin/hbase that might need TLC now that Hadoop can be 
 spotted in the wild in split-package configuration:
 {noformat}
 #If avail, add Hadoop to the CLASSPATH and to the JAVA_LIBRARY_PATH
 if [ ! -z $HADOOP_HOME ]; then
   HADOOPCPPATH=
   if [ -z $HADOOP_CONF_DIR ]; then
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} ${HADOOP_HOME}/conf)
   else
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} ${HADOOP_CONF_DIR})
   fi
   if [ `echo ${HADOOP_HOME}/hadoop-core*.jar` != 
 ${HADOOP_HOME}/hadoop-core*.jar ] ; then
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} `ls 
 ${HADOOP_HOME}/hadoop-core*.jar | head -1`)
   else
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} `ls 
 ${HADOOP_HOME}/hadoop-common*.jar | head -1`)
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} `ls 
 ${HADOOP_HOME}/hadoop-hdfs*.jar | head -1`)
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} `ls 
 ${HADOOP_HOME}/hadoop-mapred*.jar | head -1`)
   fi
 {noformat}
 There's a couple of issues with the above code:
0. HADOOP_HOME is now deprecated in Hadoop 0.23
1. the list of jar files added to the class-path should be revised
2. we need to figure out a more robust way to get the jar files that are 
 needed to the classpath (things like hadoop-mapred*.jar tend to match 
 src/test jars as well)
 Better yet, it would be useful to look into whether we can transition HBase's 
 bin/hbase onto using bin/hadoop as a launcher script instead of direct JAVA 
 invocations (Pig, Hive, Sqoop and Mahout already do that)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5241) Deletes should not mask Puts that come after it.

[
https://issues.apache.org/jira/browse/HBASE-5241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221653#comment-13221653
]

Phabricator commented on HBASE-5241:

aaiyer has commented on the revision HBASE-5241 [jira] Deletes should not mask
Puts that come after it..

INLINE COMMENTS
src/main/java/org/apache/hadoop/hbase/regionserver/Store.java:609 It seems
like going for stronger read semantics would disable the space savings we get
by storing the memstoreTS as a (1 byte) Zero value, instead of the actual value.

Perhaps, one way to get similar savings would be to store the actual
memstoreTS in the variable length encoding, after differential encoding.

Here is what I propose.
Keep track of a per-StoreFile startMemstoreTS value, that (approximately)
keeps track of the smallest memstoreTS in the file.

KV's will store the deltas such that KV's-memstoreTS =
StoreFile's-startMemstoreTS + KV-delta.

If the delta is small enough, we will only use 1 or 2 bytes for storing
it. Since we use Bytes.writeVLong: From the java docs:

if n in [-32, 127): encode in one byte with the actual value. Otherwise,
if n in [-20*2^8, 20*2^8): encode in two bytes: byte[0] = n/256 - 52;
byte[1]=n0xff. Otherwise,

REVISION DETAIL
https://reviews.facebook.net/D1731

Deletes should not mask Puts that come after it.

Key: HBASE-5241
URL: https://issues.apache.org/jira/browse/HBASE-5241
Project: HBase
Issue Type: Improvement
Reporter: Amitanand Aiyer
Assignee: Amitanand Aiyer
Attachments: HBASE-5241.D1731.1.patch, HBASE-5241.D1731.2.patch,
HBASE-5241.D1731.3.patch

Suppose that we have a delete row, and then followed by the put. The delete
row
can mask the put, unless there was a major compaction in between.
Now that we are flushing the memstoreTS to disk, along with the KVs, we
should be able
to differentiate whether or not the Put happened after the Delete and offer
better
delete semantics.
Couldn't find a pre-existing JIRA that already discusses this, so creating
one.
Seems related to https://issues.apache.org/jira/browse/HBASE-2406, but is not
quite the same.

[jira] [Updated] (HBASE-5515) Add a processRow API that supports atomic multiple reads and writes on a row


 [ 
https://issues.apache.org/jira/browse/HBASE-5515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5515:
---

Attachment: HBASE-5515.D2067.1.patch

sc requested code review of HBASE-5515 [jira] Add a processRow API that 
supports atomic multiple reads and writes on a row.
Reviewers: tedyu, dhruba, JIRA

  Add a read-modify-write row operation

  Task ID: #

  Blame Rev:

  We have modified HRegion.java internally to do some atomic row processing. It 
will be nice to have a plugable API for this.

TEST PLAN
  unit test

  Revert Plan:

  Tags:

REVISION DETAIL
  https://reviews.facebook.net/D2067

AFFECTED FILES
  src/main/java/org/apache/hadoop/hbase/client/HTable.java
  src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java
  src/main/java/org/apache/hadoop/hbase/client/RowProcessor.java
  src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
  src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
  src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
  src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
  src/test/java/org/apache/hadoop/hbase/client/TestProcessRow.java

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/4425/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.


 Add a processRow API that supports atomic multiple reads and writes on a row
 

 Key: HBASE-5515
 URL: https://issues.apache.org/jira/browse/HBASE-5515
 Project: HBase
  Issue Type: New Feature
Reporter: Scott Chen
Assignee: Scott Chen
 Attachments: HBASE-5515.D2067.1.patch


 We have modified HRegion.java internally to do some atomic row processing. It 
 will be nice to have a plugable API for this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5515) Add a processRow API that supports atomic multiple reads and writes on a row


 [ 
https://issues.apache.org/jira/browse/HBASE-5515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5515:
---

Attachment: HBASE-5515.D2067.2.patch

sc updated the revision HBASE-5515 [jira] Add a processRow API that supports 
atomic multiple reads and writes on a row.
Reviewers: tedyu, dhruba, JIRA

  Update some comments

REVISION DETAIL
  https://reviews.facebook.net/D2067

AFFECTED FILES
  src/main/java/org/apache/hadoop/hbase/client/HTable.java
  src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java
  src/main/java/org/apache/hadoop/hbase/client/RowProcessor.java
  src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
  src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
  src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
  src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
  src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
  src/test/java/org/apache/hadoop/hbase/client/TestProcessRow.java


 Add a processRow API that supports atomic multiple reads and writes on a row
 

 Key: HBASE-5515
 URL: https://issues.apache.org/jira/browse/HBASE-5515
 Project: HBase
  Issue Type: New Feature
Reporter: Scott Chen
Assignee: Scott Chen
 Attachments: HBASE-5515.D2067.1.patch, HBASE-5515.D2067.2.patch


 We have modified HRegion.java internally to do some atomic row processing. It 
 will be nice to have a plugable API for this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5286) bin/hbase's logic of adding Hadoop jar files to the classpath is fragile when presented with split packaged Hadoop 0.23 installation


[ 
https://issues.apache.org/jira/browse/HBASE-5286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221672#comment-13221672
 ] 

Hudson commented on HBASE-5286:
---

Integrated in HBase-0.92 #314 (See 
[https://builds.apache.org/job/HBase-0.92/314/])
HBASE-5286 bin/hbase's logic of adding Hadoop jar files to the classpath is 
fragile when presented with split packaged Hadoop 0.23 installation (Revision 
129)

 Result = FAILURE
stack : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/bin/hbase
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/GetJavaProperty.java


 bin/hbase's logic of adding Hadoop jar files to the classpath is fragile when 
 presented with split packaged Hadoop 0.23 installation
 

 Key: HBASE-5286
 URL: https://issues.apache.org/jira/browse/HBASE-5286
 Project: HBase
  Issue Type: Bug
  Components: scripts
Affects Versions: 0.92.0
Reporter: Roman Shaposhnik
Assignee: Roman Shaposhnik
 Fix For: 0.92.1, 0.94.0

 Attachments: HBASE-5286.patch.txt


 Here's the bit from bin/hbase that might need TLC now that Hadoop can be 
 spotted in the wild in split-package configuration:
 {noformat}
 #If avail, add Hadoop to the CLASSPATH and to the JAVA_LIBRARY_PATH
 if [ ! -z $HADOOP_HOME ]; then
   HADOOPCPPATH=
   if [ -z $HADOOP_CONF_DIR ]; then
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} ${HADOOP_HOME}/conf)
   else
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} ${HADOOP_CONF_DIR})
   fi
   if [ `echo ${HADOOP_HOME}/hadoop-core*.jar` != 
 ${HADOOP_HOME}/hadoop-core*.jar ] ; then
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} `ls 
 ${HADOOP_HOME}/hadoop-core*.jar | head -1`)
   else
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} `ls 
 ${HADOOP_HOME}/hadoop-common*.jar | head -1`)
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} `ls 
 ${HADOOP_HOME}/hadoop-hdfs*.jar | head -1`)
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} `ls 
 ${HADOOP_HOME}/hadoop-mapred*.jar | head -1`)
   fi
 {noformat}
 There's a couple of issues with the above code:
0. HADOOP_HOME is now deprecated in Hadoop 0.23
1. the list of jar files added to the class-path should be revised
2. we need to figure out a more robust way to get the jar files that are 
 needed to the classpath (things like hadoop-mapred*.jar tend to match 
 src/test jars as well)
 Better yet, it would be useful to look into whether we can transition HBase's 
 bin/hbase onto using bin/hadoop as a launcher script instead of direct JAVA 
 invocations (Pig, Hive, Sqoop and Mahout already do that)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5515) Add a processRow API that supports atomic multiple reads and writes on a row


 [ 
https://issues.apache.org/jira/browse/HBASE-5515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5515:
---

Attachment: HBASE-5515.D2067.3.patch

sc updated the revision HBASE-5515 [jira] Add a processRow API that supports 
atomic multiple reads and writes on a row.
Reviewers: tedyu, dhruba, JIRA

  More comments change

REVISION DETAIL
  https://reviews.facebook.net/D2067

AFFECTED FILES
  src/main/java/org/apache/hadoop/hbase/client/HTable.java
  src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java
  src/main/java/org/apache/hadoop/hbase/client/RowProcessor.java
  src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
  src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
  src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
  src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
  src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
  src/test/java/org/apache/hadoop/hbase/client/TestProcessRow.java


 Add a processRow API that supports atomic multiple reads and writes on a row
 

 Key: HBASE-5515
 URL: https://issues.apache.org/jira/browse/HBASE-5515
 Project: HBase
  Issue Type: New Feature
Reporter: Scott Chen
Assignee: Scott Chen
 Attachments: HBASE-5515.D2067.1.patch, HBASE-5515.D2067.2.patch, 
 HBASE-5515.D2067.3.patch


 We have modified HRegion.java internally to do some atomic row processing. It 
 will be nice to have a plugable API for this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5507) ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not use ByteBuffer correctly

2012-03-03 Thread Scott Chen (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Chen updated HBASE-5507:
--

Summary: ThriftServerRunner.HbaseHandler.getRegionInfo() and 
getTableRegions() do not use ByteBuffer correctly  (was: 
ThriftServerRunner.HbaseHandler.getRegionInfo() gets error for the case of 
(framed, compact) = (true, false))

 ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not 
 use ByteBuffer correctly
 -

 Key: HBASE-5507
 URL: https://issues.apache.org/jira/browse/HBASE-5507
 Project: HBase
  Issue Type: Bug
  Components: thrift
Reporter: Scott Chen
Assignee: Scott Chen

 We observed that when with framed transport option. The thrift call 
 ThriftServerRunner.HbaseHandler.getRegionInfo() receives corrupted parameter 
 (some garbage string attached to the beginning). This may be a thrift bug 
 requires further investigation.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5371) Introduce AccessControllerProtocol.checkPermissions(Permission[] permissons) API

2012-03-03 Thread Andrew Purtell (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221701#comment-13221701
 ] 

Andrew Purtell commented on HBASE-5371:
---

+1 apologies for the delay

 Introduce AccessControllerProtocol.checkPermissions(Permission[] permissons) 
 API
 

 Key: HBASE-5371
 URL: https://issues.apache.org/jira/browse/HBASE-5371
 Project: HBase
  Issue Type: Sub-task
  Components: security
Affects Versions: 0.92.1, 0.94.0
Reporter: Enis Soztutar
Assignee: Enis Soztutar
 Fix For: 0.94.0

 Attachments: HBASE-5371-addendum_v1.patch, HBASE-5371_v2.patch, 
 HBASE-5371_v3-noprefix.patch, HBASE-5371_v3.patch


 We need to introduce something like 
 AccessControllerProtocol.checkPermissions(Permission[] permissions) API, so 
 that clients can check access rights before carrying out the operations. We 
 need this kind of operation for HCATALOG-245, which introduces authorization 
 providers for hbase over hcat. We cannot use getUserPermissions() since it 
 requires ADMIN permissions on the global/table level.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5507) ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not use ByteBuffer correctly


 [ 
https://issues.apache.org/jira/browse/HBASE-5507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5507:
---

Attachment: HBASE-5507.D2073.1.patch

sc requested code review of HBASE-5507 [jira] 
ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not 
use ByteBuffer correctly.
Reviewers: tedyu, dhruba, JIRA

  ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not 
use ByteBuffer correctly

  Task ID: #

  Blame Rev:

  We observed that when with framed transport option. The thrift call 
ThriftServerRunner.HbaseHandler.getRegionInfo() receives corrupted parameter 
(some garbage string attached to the beginning). This may be a thrift bug 
requires further investigation.

TEST PLAN
  unit test

  Revert Plan:

  Tags:

REVISION DETAIL
  https://reviews.facebook.net/D2073

AFFECTED FILES
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java
  src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
  src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
  src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/4443/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.


 ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not 
 use ByteBuffer correctly
 -

 Key: HBASE-5507
 URL: https://issues.apache.org/jira/browse/HBASE-5507
 Project: HBase
  Issue Type: Bug
  Components: thrift
Reporter: Scott Chen
Assignee: Scott Chen
 Attachments: HBASE-5507.D2073.1.patch


 We observed that when with framed transport option. The thrift call 
 ThriftServerRunner.HbaseHandler.getRegionInfo() receives corrupted parameter 
 (some garbage string attached to the beginning). This may be a thrift bug 
 requires further investigation.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5371) Introduce AccessControllerProtocol.checkPermissions(Permission[] permissons) API


[ 
https://issues.apache.org/jira/browse/HBASE-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221708#comment-13221708
 ] 

Zhihong Yu commented on HBASE-5371:
---

Addendum integrated into 0.94 and TRUNK.

Waiting for Stack's confirmation on when to integrate into 0.92

 Introduce AccessControllerProtocol.checkPermissions(Permission[] permissons) 
 API
 

 Key: HBASE-5371
 URL: https://issues.apache.org/jira/browse/HBASE-5371
 Project: HBase
  Issue Type: Sub-task
  Components: security
Affects Versions: 0.92.1, 0.94.0
Reporter: Enis Soztutar
Assignee: Enis Soztutar
 Fix For: 0.94.0

 Attachments: HBASE-5371-addendum_v1.patch, HBASE-5371_v2.patch, 
 HBASE-5371_v3-noprefix.patch, HBASE-5371_v3.patch


 We need to introduce something like 
 AccessControllerProtocol.checkPermissions(Permission[] permissions) API, so 
 that clients can check access rights before carrying out the operations. We 
 need this kind of operation for HCATALOG-245, which introduces authorization 
 providers for hbase over hcat. We cannot use getUserPermissions() since it 
 requires ADMIN permissions on the global/table level.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5241) Deletes should not mask Puts that come after it.

[
https://issues.apache.org/jira/browse/HBASE-5241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221710#comment-13221710
]

Phabricator commented on HBASE-5241:

tedyu has commented on the revision HBASE-5241 [jira] Deletes should not mask
Puts that come after it..

INLINE COMMENTS
src/main/java/org/apache/hadoop/hbase/regionserver/Store.java:609 I think
this suggestion is a good idea.

REVISION DETAIL
https://reviews.facebook.net/D1731

Deletes should not mask Puts that come after it.

[jira] [Commented] (HBASE-5507) ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not use ByteBuffer correctly


[ 
https://issues.apache.org/jira/browse/HBASE-5507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221711#comment-13221711
 ] 

Phabricator commented on HBASE-5507:


tedyu has commented on the revision HBASE-5507 [jira] 
ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not 
use ByteBuffer correctly.

INLINE COMMENTS
  src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java:83 
This comment can be removed, right ?

REVISION DETAIL
  https://reviews.facebook.net/D2073


 ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not 
 use ByteBuffer correctly
 -

 Key: HBASE-5507
 URL: https://issues.apache.org/jira/browse/HBASE-5507
 Project: HBase
  Issue Type: Bug
  Components: thrift
Reporter: Scott Chen
Assignee: Scott Chen
 Attachments: HBASE-5507.D2073.1.patch


 We observed that when with framed transport option. The thrift call 
 ThriftServerRunner.HbaseHandler.getRegionInfo() receives corrupted parameter 
 (some garbage string attached to the beginning). This may be a thrift bug 
 requires further investigation.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5507) ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not use ByteBuffer correctly

2012-03-03 Thread Zhihong Yu (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5507:
--

Hadoop Flags: Reviewed
  Status: Patch Available  (was: Open)

 ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not 
 use ByteBuffer correctly
 -

 Key: HBASE-5507
 URL: https://issues.apache.org/jira/browse/HBASE-5507
 Project: HBase
  Issue Type: Bug
  Components: thrift
Reporter: Scott Chen
Assignee: Scott Chen
 Attachments: HBASE-5507.D2073.1.patch


 We observed that when with framed transport option. The thrift call 
 ThriftServerRunner.HbaseHandler.getRegionInfo() receives corrupted parameter 
 (some garbage string attached to the beginning). This may be a thrift bug 
 requires further investigation.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5515) Add a processRow API that supports atomic multiple reads and writes on a row

2012-03-03 Thread Zhihong Yu (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5515:
--

Status: Patch Available  (was: Open)

 Add a processRow API that supports atomic multiple reads and writes on a row
 

 Key: HBASE-5515
 URL: https://issues.apache.org/jira/browse/HBASE-5515
 Project: HBase
  Issue Type: New Feature
Reporter: Scott Chen
Assignee: Scott Chen
 Attachments: HBASE-5515.D2067.1.patch, HBASE-5515.D2067.2.patch, 
 HBASE-5515.D2067.3.patch


 We have modified HRegion.java internally to do some atomic row processing. It 
 will be nice to have a plugable API for this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5515) Add a processRow API that supports atomic multiple reads and writes on a row


[ 
https://issues.apache.org/jira/browse/HBASE-5515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221714#comment-13221714
 ] 

Phabricator commented on HBASE-5515:


tedyu has commented on the revision HBASE-5515 [jira] Add a processRow API 
that supports atomic multiple reads and writes on a row.

  The difference between this API and mutateRow() is that RowMutations don't 
cover scan, right ?

INLINE COMMENTS
  src/main/java/org/apache/hadoop/hbase/client/RowProcessor.java:28 There is 
only one RowProcessor.RowScanner passed to process().
  So scan should be singular.
  src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java:268 I think 
this should be moved to after GENERIC_ARRAY_CODE, line 272

REVISION DETAIL
  https://reviews.facebook.net/D2067


 Add a processRow API that supports atomic multiple reads and writes on a row
 

 Key: HBASE-5515
 URL: https://issues.apache.org/jira/browse/HBASE-5515
 Project: HBase
  Issue Type: New Feature
Reporter: Scott Chen
Assignee: Scott Chen
 Attachments: HBASE-5515.D2067.1.patch, HBASE-5515.D2067.2.patch, 
 HBASE-5515.D2067.3.patch


 We have modified HRegion.java internally to do some atomic row processing. It 
 will be nice to have a plugable API for this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5515) Add a processRow API that supports atomic multiple reads and writes on a row

[
https://issues.apache.org/jira/browse/HBASE-5515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221718#comment-13221718
]

Hadoop QA commented on HBASE-5515:
--

-1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12516953/HBASE-5515.D2067.3.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 2 new or modified tests.

-1 javadoc. The javadoc tool appears to have generated -126 warning
messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 154 new Findbugs (version
1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.io.TestHeapSize
org.apache.hadoop.hbase.io.TestHbaseObjectWritable

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1085//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1085//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1085//console

This message is automatically generated.

Add a processRow API that supports atomic multiple reads and writes on a row

Key: HBASE-5515
URL: https://issues.apache.org/jira/browse/HBASE-5515
Project: HBase
Issue Type: New Feature
Reporter: Scott Chen
Assignee: Scott Chen
Attachments: HBASE-5515.D2067.1.patch, HBASE-5515.D2067.2.patch,
HBASE-5515.D2067.3.patch

We have modified HRegion.java internally to do some atomic row processing. It
will be nice to have a plugable API for this.

[jira] [Commented] (HBASE-5371) Introduce AccessControllerProtocol.checkPermissions(Permission[] permissons) API


[ 
https://issues.apache.org/jira/browse/HBASE-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221722#comment-13221722
 ] 

Hudson commented on HBASE-5371:
---

Integrated in HBase-0.94 #12 (See 
[https://builds.apache.org/job/HBase-0.94/12/])
HBASE-5371 Introduce AccessControllerProtocol.checkPermissions(Permission[] 
permissons) API, addendum (Enis) (Revision 1296710)

 Result = SUCCESS
tedyu : 
Files : 
* 
/hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
* 
/hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java


 Introduce AccessControllerProtocol.checkPermissions(Permission[] permissons) 
 API
 

 Key: HBASE-5371
 URL: https://issues.apache.org/jira/browse/HBASE-5371
 Project: HBase
  Issue Type: Sub-task
  Components: security
Affects Versions: 0.92.1, 0.94.0
Reporter: Enis Soztutar
Assignee: Enis Soztutar
 Fix For: 0.94.0

 Attachments: HBASE-5371-addendum_v1.patch, HBASE-5371_v2.patch, 
 HBASE-5371_v3-noprefix.patch, HBASE-5371_v3.patch


 We need to introduce something like 
 AccessControllerProtocol.checkPermissions(Permission[] permissions) API, so 
 that clients can check access rights before carrying out the operations. We 
 need this kind of operation for HCATALOG-245, which introduces authorization 
 providers for hbase over hcat. We cannot use getUserPermissions() since it 
 requires ADMIN permissions on the global/table level.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5515) Add a processRow API that supports atomic multiple reads and writes on a row


[ 
https://issues.apache.org/jira/browse/HBASE-5515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221723#comment-13221723
 ] 

Phabricator commented on HBASE-5515:


tedyu has commented on the revision HBASE-5515 [jira] Add a processRow API 
that supports atomic multiple reads and writes on a row.

  Only small tests were run in 
https://builds.apache.org/job/PreCommit-HBASE-Build/1085

INLINE COMMENTS
  src/main/java/org/apache/hadoop/hbase/client/RowProcessor.java:28 Discard 
previous comment.
  The scanner can be used to perform multiple scans.
  src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java:268 Please 
add an assertion at the end of TestHbaseObjectWritable.testGetClassCode() for 
RowProcessor.

  Change the ordinal to 83 in TestHbaseObjectWritable.testGetNextObjectCode()
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:4376 1 
ClassSize.REFERENCE should be added to HRegion.FIXED_OVERHEAD

REVISION DETAIL
  https://reviews.facebook.net/D2067


 Add a processRow API that supports atomic multiple reads and writes on a row
 

 Key: HBASE-5515
 URL: https://issues.apache.org/jira/browse/HBASE-5515
 Project: HBase
  Issue Type: New Feature
Reporter: Scott Chen
Assignee: Scott Chen
 Attachments: HBASE-5515.D2067.1.patch, HBASE-5515.D2067.2.patch, 
 HBASE-5515.D2067.3.patch


 We have modified HRegion.java internally to do some atomic row processing. It 
 will be nice to have a plugable API for this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5507) ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not use ByteBuffer correctly

[
https://issues.apache.org/jira/browse/HBASE-5507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221725#comment-13221725
]

Hadoop QA commented on HBASE-5507:
--

-1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12516961/HBASE-5507.D2073.1.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 6 new or modified tests.

-1 javadoc. The javadoc tool appears to have generated -129 warning
messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 154 new Findbugs (version
1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.thrift.TestThriftServer
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1084//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1084//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1084//console

This message is automatically generated.

ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not
use ByteBuffer correctly
-

Key: HBASE-5507
URL: https://issues.apache.org/jira/browse/HBASE-5507
Project: HBase
Issue Type: Bug
Components: thrift
Reporter: Scott Chen
Assignee: Scott Chen
Attachments: HBASE-5507.D2073.1.patch

We observed that when with framed transport option. The thrift call
ThriftServerRunner.HbaseHandler.getRegionInfo() receives corrupted parameter
(some garbage string attached to the beginning). This may be a thrift bug
requires further investigation.

[jira] [Commented] (HBASE-5515) Add a processRow API that supports atomic multiple reads and writes on a row


[ 
https://issues.apache.org/jira/browse/HBASE-5515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221726#comment-13221726
 ] 

Phabricator commented on HBASE-5515:


tedyu has commented on the revision HBASE-5515 [jira] Add a processRow API 
that supports atomic multiple reads and writes on a row.

INLINE COMMENTS
  src/main/java/org/apache/hadoop/hbase/client/RowProcessor.java:31 Please tag 
this interface @InterfaceAudience.Public

REVISION DETAIL
  https://reviews.facebook.net/D2067


 Add a processRow API that supports atomic multiple reads and writes on a row
 

 Key: HBASE-5515
 URL: https://issues.apache.org/jira/browse/HBASE-5515
 Project: HBase
  Issue Type: New Feature
Reporter: Scott Chen
Assignee: Scott Chen
 Attachments: HBASE-5515.D2067.1.patch, HBASE-5515.D2067.2.patch, 
 HBASE-5515.D2067.3.patch


 We have modified HRegion.java internally to do some atomic row processing. It 
 will be nice to have a plugable API for this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2038) Coprocessors: Region level indexing

[
https://issues.apache.org/jira/browse/HBASE-2038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221759#comment-13221759
]

Lars Hofhansl commented on HBASE-2038:
--

@Anoop: We can certainly try to expose this at the RegionScanner level.
Although I feel it might actually be harder than you think, as the seeking is
dealt with on a store basis, and we do not want to inhibit the ability to deal
with Stores in parallel in the future.
RegionScanner.seek would have to go through all Stores and for each Store seek
the MemstoreScanner and all StoreFileScanners. Seeking this way across stores
is only valid if we seek on row boundaries (each Store - i.e. column family -
has it's own set of columns, which could even have the same names between
stores).

@HBASE-5521: I started working on that, but I am starting to question the
usefulness.
A filter is per KeyValue (at least the method that allows for seeking). So,
many KeyValues flow through the Filter for a single row, and the filter needs
to seek separately for each ColumnFamily (as explained above and on the mailing
list).
So the gain from this would be fairly minimal (which I guess is why we do not
have this).
For example a row with many column would need to issue many INCLUDE's and only
for the last KeyVakue (and how would it know it's the last?) issue
INCLUDE_AND_SEEK...

Coprocessors: Region level indexing
---

Key: HBASE-2038
URL: https://issues.apache.org/jira/browse/HBASE-2038
Project: HBase
Issue Type: New Feature
Components: coprocessors
Reporter: Andrew Purtell
Priority: Minor

HBASE-2037 is a good candidate to be done as coprocessor. It also serve as a
good goalpost for coprocessor environment design -- there should be enough of
it so region level indexing can be reimplemented as a coprocessor without any
loss of functionality.

[jira] [Commented] (HBASE-5512) Add support for INCLUDE_AND_SEEK_USING_HINT


[ 
https://issues.apache.org/jira/browse/HBASE-5512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221760#comment-13221760
 ] 

Lars Hofhansl commented on HBASE-5512:
--

@Anoop: Good observation :) (As I said I just needed to store the partial patch 
somewhere).
@Ram: Sure. Would you prefer to continue to work on this?

Also see last comment on HBASE-2038. Not entirely sure how much mileage we get 
from this.

 Add support for INCLUDE_AND_SEEK_USING_HINT
 ---

 Key: HBASE-5512
 URL: https://issues.apache.org/jira/browse/HBASE-5512
 Project: HBase
  Issue Type: Improvement
Reporter: Zhihong Yu
Assignee: Lars Hofhansl
 Attachments: 5512.txt


 This came up from HBASE-2038
 From Anoop:
 - What we wanted from the filter is include a row and then seek to the next 
 row which we are interested in. I cant see such a facility with our Filter 
 right now. Correct me if I am wrong. So suppose we already seeked to one row 
 and this need to be included in the result, then the Filter should return 
 INCLUDE. Then when the next next() call happens, then only we can return a 
 SEEK_USING_HINT. So one extra row reading is needed. This might create even 
 one unwanted HFileBlock fetch (who knows).
 Can we add reseek() at higher level?
 From Lars:
 Yep, for that we'd need to add INCLUDE_AND_SEEK_USING_HINT (similar to the 
 INCLUDE_AND_SEEK_NEXT_ROW that we already have). Shouldn't be hard to add, 
 I'm happy to do that, if that's the route we want to go with this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5507) ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not use ByteBuffer correctly


[ 
https://issues.apache.org/jira/browse/HBASE-5507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221775#comment-13221775
 ] 

Phabricator commented on HBASE-5507:


sc has commented on the revision HBASE-5507 [jira] 
ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not 
use ByteBuffer correctly.

INLINE COMMENTS
  src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java:83 
Yes, you're right. Sorry for leaving this here.

REVISION DETAIL
  https://reviews.facebook.net/D2073


 ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not 
 use ByteBuffer correctly
 -

 Key: HBASE-5507
 URL: https://issues.apache.org/jira/browse/HBASE-5507
 Project: HBase
  Issue Type: Bug
  Components: thrift
Reporter: Scott Chen
Assignee: Scott Chen
 Attachments: HBASE-5507.D2073.1.patch


 We observed that when with framed transport option. The thrift call 
 ThriftServerRunner.HbaseHandler.getRegionInfo() receives corrupted parameter 
 (some garbage string attached to the beginning). This may be a thrift bug 
 requires further investigation.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5507) ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not use ByteBuffer correctly


 [ 
https://issues.apache.org/jira/browse/HBASE-5507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5507:
---

Attachment: HBASE-5507.D2073.2.patch

sc updated the revision HBASE-5507 [jira] 
ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not 
use ByteBuffer correctly.
Reviewers: tedyu, dhruba, JIRA

  Addressed Ted's review comments, thanks.

REVISION DETAIL
  https://reviews.facebook.net/D2073

AFFECTED FILES
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java
  src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
  src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
  src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java


 ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not 
 use ByteBuffer correctly
 -

 Key: HBASE-5507
 URL: https://issues.apache.org/jira/browse/HBASE-5507
 Project: HBase
  Issue Type: Bug
  Components: thrift
Reporter: Scott Chen
Assignee: Scott Chen
 Attachments: HBASE-5507.D2073.1.patch, HBASE-5507.D2073.2.patch


 We observed that when with framed transport option. The thrift call 
 ThriftServerRunner.HbaseHandler.getRegionInfo() receives corrupted parameter 
 (some garbage string attached to the beginning). This may be a thrift bug 
 requires further investigation.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5515) Add a processRow API that supports atomic multiple reads and writes on a row


[ 
https://issues.apache.org/jira/browse/HBASE-5515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221777#comment-13221777
 ] 

Phabricator commented on HBASE-5515:


sc has commented on the revision HBASE-5515 [jira] Add a processRow API that 
supports atomic multiple reads and writes on a row.

  Ted: Thanks for the review. I will update this soon.

INLINE COMMENTS
  src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java:268 Thanks! 
good catch
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:4376 Good 
catch!

REVISION DETAIL
  https://reviews.facebook.net/D2067


 Add a processRow API that supports atomic multiple reads and writes on a row
 

 Key: HBASE-5515
 URL: https://issues.apache.org/jira/browse/HBASE-5515
 Project: HBase
  Issue Type: New Feature
Reporter: Scott Chen
Assignee: Scott Chen
 Attachments: HBASE-5515.D2067.1.patch, HBASE-5515.D2067.2.patch, 
 HBASE-5515.D2067.3.patch


 We have modified HRegion.java internally to do some atomic row processing. It 
 will be nice to have a plugable API for this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5512) Add support for INCLUDE_AND_SEEK_USING_HINT

2012-03-03 Thread Lars Hofhansl (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5512:
-

Attachment: 5512-v2.txt

This version should work.
(but see caveats raised before)

 Add support for INCLUDE_AND_SEEK_USING_HINT
 ---

 Key: HBASE-5512
 URL: https://issues.apache.org/jira/browse/HBASE-5512
 Project: HBase
  Issue Type: Improvement
Reporter: Zhihong Yu
Assignee: Lars Hofhansl
 Attachments: 5512-v2.txt, 5512.txt


 This came up from HBASE-2038
 From Anoop:
 - What we wanted from the filter is include a row and then seek to the next 
 row which we are interested in. I cant see such a facility with our Filter 
 right now. Correct me if I am wrong. So suppose we already seeked to one row 
 and this need to be included in the result, then the Filter should return 
 INCLUDE. Then when the next next() call happens, then only we can return a 
 SEEK_USING_HINT. So one extra row reading is needed. This might create even 
 one unwanted HFileBlock fetch (who knows).
 Can we add reseek() at higher level?
 From Lars:
 Yep, for that we'd need to add INCLUDE_AND_SEEK_USING_HINT (similar to the 
 INCLUDE_AND_SEEK_NEXT_ROW that we already have). Shouldn't be hard to add, 
 I'm happy to do that, if that's the route we want to go with this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5515) Add a processRow API that supports atomic multiple reads and writes on a row


[ 
https://issues.apache.org/jira/browse/HBASE-5515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221780#comment-13221780
 ] 

Phabricator commented on HBASE-5515:


sc has commented on the revision HBASE-5515 [jira] Add a processRow API that 
supports atomic multiple reads and writes on a row.

  Ted: Yes, this API allows read-modify-write atomically. Also you can perform 
multiple reads with dependency on previous results (for example, querying 
friends of friends in the unit test).

REVISION DETAIL
  https://reviews.facebook.net/D2067


 Add a processRow API that supports atomic multiple reads and writes on a row
 

 Key: HBASE-5515
 URL: https://issues.apache.org/jira/browse/HBASE-5515
 Project: HBase
  Issue Type: New Feature
Reporter: Scott Chen
Assignee: Scott Chen
 Attachments: HBASE-5515.D2067.1.patch, HBASE-5515.D2067.2.patch, 
 HBASE-5515.D2067.3.patch, HBASE-5515.D2067.4.patch


 We have modified HRegion.java internally to do some atomic row processing. It 
 will be nice to have a plugable API for this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5515) Add a processRow API that supports atomic multiple reads and writes on a row


 [ 
https://issues.apache.org/jira/browse/HBASE-5515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5515:
---

Attachment: HBASE-5515.D2067.4.patch

sc updated the revision HBASE-5515 [jira] Add a processRow API that supports 
atomic multiple reads and writes on a row.
Reviewers: tedyu, dhruba, JIRA

  Addressed Ted's comments

REVISION DETAIL
  https://reviews.facebook.net/D2067

AFFECTED FILES
  src/main/java/org/apache/hadoop/hbase/client/HTable.java
  src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java
  src/main/java/org/apache/hadoop/hbase/client/RowProcessor.java
  src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
  src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
  src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
  src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
  src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
  src/test/java/org/apache/hadoop/hbase/client/TestProcessRow.java
  src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java


 Add a processRow API that supports atomic multiple reads and writes on a row
 

 Key: HBASE-5515
 URL: https://issues.apache.org/jira/browse/HBASE-5515
 Project: HBase
  Issue Type: New Feature
Reporter: Scott Chen
Assignee: Scott Chen
 Attachments: HBASE-5515.D2067.1.patch, HBASE-5515.D2067.2.patch, 
 HBASE-5515.D2067.3.patch, HBASE-5515.D2067.4.patch


 We have modified HRegion.java internally to do some atomic row processing. It 
 will be nice to have a plugable API for this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5515) Add a processRow API that supports atomic multiple reads and writes on a row

[
https://issues.apache.org/jira/browse/HBASE-5515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221781#comment-13221781
]

Hadoop QA commented on HBASE-5515:
--

-1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12516971/HBASE-5515.D2067.4.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 5 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

-1 javac. The patch appears to cause mvn compile goal to fail.

-1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1087//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1087//console

This message is automatically generated.

Add a processRow API that supports atomic multiple reads and writes on a row

We have modified HRegion.java internally to do some atomic row processing. It
will be nice to have a plugable API for this.

[jira] [Commented] (HBASE-5507) ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not use ByteBuffer correctly

[
https://issues.apache.org/jira/browse/HBASE-5507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221782#comment-13221782
]

Hadoop QA commented on HBASE-5507:
--

-1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12516969/HBASE-5507.D2073.2.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 6 new or modified tests.

-1 javadoc. The javadoc tool appears to have generated -129 warning
messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 154 new Findbugs (version
1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1086//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1086//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1086//console

This message is automatically generated.

ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not
use ByteBuffer correctly
-

Key: HBASE-5507
URL: https://issues.apache.org/jira/browse/HBASE-5507
Project: HBase
Issue Type: Bug
Components: thrift
Reporter: Scott Chen
Assignee: Scott Chen
Attachments: HBASE-5507.D2073.1.patch, HBASE-5507.D2073.2.patch

[jira] [Commented] (HBASE-5515) Add a processRow API that supports atomic multiple reads and writes on a row


[ 
https://issues.apache.org/jira/browse/HBASE-5515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221786#comment-13221786
 ] 

Phabricator commented on HBASE-5515:


lhofhansl has commented on the revision HBASE-5515 [jira] Add a processRow API 
that supports atomic multiple reads and writes on a row.

  We're getting a lot of similar but slightly different APIs now.

  This is not a for a fixed use case, but basically this a command pattern 
where we pass an arbitrary command to the server execution there.

  The RowProcessor implementation needs to exist in the RegionServer's 
classpath, right?

  Like atomic multi row mutations, maybe this should only be accessible to a 
Coprocessor endpoint (see final resolution for HBASE-5229), and not to the 
public HTable API.

  Pie in the sky: It would be nice if the command itself could be implemented 
as a coprocessor endpoint (would need to expose locks and mvcc to 
region-coprocessors, so maybe that's overkill).


REVISION DETAIL
  https://reviews.facebook.net/D2067


 Add a processRow API that supports atomic multiple reads and writes on a row
 

 Key: HBASE-5515
 URL: https://issues.apache.org/jira/browse/HBASE-5515
 Project: HBase
  Issue Type: New Feature
Reporter: Scott Chen
Assignee: Scott Chen
 Attachments: HBASE-5515.D2067.1.patch, HBASE-5515.D2067.2.patch, 
 HBASE-5515.D2067.3.patch, HBASE-5515.D2067.4.patch


 We have modified HRegion.java internally to do some atomic row processing. It 
 will be nice to have a plugable API for this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5509) MR based copier for copying HFiles (trunk version)

[
https://issues.apache.org/jira/browse/HBASE-5509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221787#comment-13221787
]

Lars Hofhansl commented on HBASE-5509:
--

bq. For sameFile(), I think false should be returned for dest file in the
following case:{code}
+ //return true if checksum is not supported
+ //(i.e. some of the checksums is null)
{code}

Not sure I agree with that. The method comment says that if either FS does not
support checksums and that check is ignored. I.e. if any of the earlier tests
(like size comparison) did not flag the files as different they are considered
equal).

MR based copier for copying HFiles (trunk version)
--

Key: HBASE-5509
URL: https://issues.apache.org/jira/browse/HBASE-5509
Project: HBase
Issue Type: Sub-task
Components: documentation, regionserver
Reporter: Karthik Ranganathan
Assignee: Lars Hofhansl
Fix For: 0.94.0, 0.96.0

Attachments: 5509.txt

This copier is a modification of the distcp tool in HDFS. It does the
following:
1. List out all the regions in the HBase cluster for the required table
2. Write the above out to a file
3. Each mapper
3.1 lists all the HFiles for a given region by querying the regionserver
3.2 copies all the HFiles
3.3 outputs success if the copy succeeded, failure otherwise. Failed
regions are retried in another loop
4. Mappers are placed on nodes which have maximum locality for a given region
to speed up copying

[jira] [Commented] (HBASE-5507) ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not use ByteBuffer correctly


[ 
https://issues.apache.org/jira/browse/HBASE-5507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221788#comment-13221788
 ] 

Zhihong Yu commented on HBASE-5507:
---

TestThriftServer failed twice as reported by Hadoop QA.

 ThriftServerRunner.HbaseHandler.getRegionInfo() and getTableRegions() do not 
 use ByteBuffer correctly
 -

 Key: HBASE-5507
 URL: https://issues.apache.org/jira/browse/HBASE-5507
 Project: HBase
  Issue Type: Bug
  Components: thrift
Reporter: Scott Chen
Assignee: Scott Chen
 Attachments: HBASE-5507.D2073.1.patch, HBASE-5507.D2073.2.patch


 We observed that when with framed transport option. The thrift call 
 ThriftServerRunner.HbaseHandler.getRegionInfo() receives corrupted parameter 
 (some garbage string attached to the beginning). This may be a thrift bug 
 requires further investigation.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5509) MR based copier for copying HFiles (trunk version)

[
https://issues.apache.org/jira/browse/HBASE-5509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221789#comment-13221789
]

Lars Hofhansl commented on HBASE-5509:
--

bq. Is it possible to make the src and dst comply to same data type ? Either
FileStatus or Path.

It is. It means the code in SnapshotMR would be slightly less readable. On the
other hand we'd do fewer RPC to get the file system status. Also requires
sameFile to be package private, otherwise we need to double check the file
here. I'll do that and then we can decide.

MR based copier for copying HFiles (trunk version)
--

Attachments: 5509.txt

[jira] [Commented] (HBASE-5509) MR based copier for copying HFiles (trunk version)

[
https://issues.apache.org/jira/browse/HBASE-5509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221796#comment-13221796
]

Lars Hofhansl commented on HBASE-5509:
--

bq. For getStoreFileList():{code}
+ * @param families
+ * a comma separated list of column families for which we need to
{code} I think ListString may be better data type for families parameter.
This would make this method more general in that

Families is taken from the command line; it might be (1) an indicator for all
CFs, or (2) a list of specific CFs. In the former case we cannot get the list
of CFs before we know the RegionServer, which only happens later.

Again, this is not my code. I can refactor and have extra command line flags
and extra code for this.

MR based copier for copying HFiles (trunk version)
--

Attachments: 5509.txt

[jira] [Commented] (HBASE-2038) Coprocessors: Region level indexing

2012-03-03 Thread Anoop Sam John (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-2038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221797#comment-13221797
]

Anoop Sam John commented on HBASE-2038:
---

{quote}
@HBASE-5521: I started working on that, but I am starting to question the
usefulness.
A filter is per KeyValue (at least the method that allows for seeking). So,
many KeyValues flow through the Filter for a single row, and the filter needs
to seek separately for each ColumnFamily (as explained above and on the mailing
list).
So the gain from this would be fairly minimal (which I guess is why we do not
have this).
For example a row with many column would need to issue many INCLUDE's and only
for the last KeyVakue (and how would it know it's the last?) issue
INCLUDE_AND_SEEK..
{quote}

Lars, I was also thinking on this yesterday after seeing the patch. I wanted
to give a test case try run before commenting :)

Regarding you 1st comment, In our above discussion scenario of seek() we need a
row boundary seek.. Yes all the stores ( memstore and all store files in that
store) need to get seeked to needed point. Let me see more on this on Monday.
we had done small changes and tested this once. I mean we were able to seek to
row boundaries.

Thanks a lot Lars for your work and suggestion

@Ram: Yes we can file a Jira for co processor support for next( int nbrows)?

Coprocessors: Region level indexing
---

Key: HBASE-2038
URL: https://issues.apache.org/jira/browse/HBASE-2038
Project: HBase
Issue Type: New Feature
Components: coprocessors
Reporter: Andrew Purtell
Priority: Minor

[jira] [Updated] (HBASE-5509) MR based copier for copying HFiles (trunk version)

2012-03-03 Thread Lars Hofhansl (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5509:
-

Attachment: 5509-v2.txt

Addressed some of the format issues raised.

Another thought: It would be great if files outputted by SnapshotMR could be 
used with LoadIncrementalHFiles.

 MR based copier for copying HFiles (trunk version)
 --

 Key: HBASE-5509
 URL: https://issues.apache.org/jira/browse/HBASE-5509
 Project: HBase
  Issue Type: Sub-task
  Components: documentation, regionserver
Reporter: Karthik Ranganathan
Assignee: Lars Hofhansl
 Fix For: 0.94.0, 0.96.0

 Attachments: 5509-v2.txt, 5509.txt


 This copier is a modification of the distcp tool in HDFS. It does the 
 following:
 1. List out all the regions in the HBase cluster for the required table
 2. Write the above out to a file
 3. Each mapper 
3.1 lists all the HFiles for a given region by querying the regionserver
3.2 copies all the HFiles
3.3 outputs success if the copy succeeded, failure otherwise. Failed 
 regions are retried in another loop
 4. Mappers are placed on nodes which have maximum locality for a given region 
 to speed up copying

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5286) bin/hbase's logic of adding Hadoop jar files to the classpath is fragile when presented with split packaged Hadoop 0.23 installation


[ 
https://issues.apache.org/jira/browse/HBASE-5286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221799#comment-13221799
 ] 

Hudson commented on HBASE-5286:
---

Integrated in HBase-TRUNK-security #128 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/128/])
HBASE-5286 bin/hbase's logic of adding Hadoop jar files to the classpath is 
fragile when presented with split packaged Hadoop 0.23 installation (Revision 
1296663)
HBASE-5286 bin/hbase's logic of adding Hadoop jar files to the classpath is 
fragile when presented with split packaged Hadoop 0.23 installation (Revision 
1296661)

 Result = FAILURE
stack : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/GetJavaProperty.java

stack : 
Files : 
* /hbase/trunk/bin/hbase


 bin/hbase's logic of adding Hadoop jar files to the classpath is fragile when 
 presented with split packaged Hadoop 0.23 installation
 

 Key: HBASE-5286
 URL: https://issues.apache.org/jira/browse/HBASE-5286
 Project: HBase
  Issue Type: Bug
  Components: scripts
Affects Versions: 0.92.0
Reporter: Roman Shaposhnik
Assignee: Roman Shaposhnik
 Fix For: 0.92.1, 0.94.0

 Attachments: HBASE-5286.patch.txt


 Here's the bit from bin/hbase that might need TLC now that Hadoop can be 
 spotted in the wild in split-package configuration:
 {noformat}
 #If avail, add Hadoop to the CLASSPATH and to the JAVA_LIBRARY_PATH
 if [ ! -z $HADOOP_HOME ]; then
   HADOOPCPPATH=
   if [ -z $HADOOP_CONF_DIR ]; then
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} ${HADOOP_HOME}/conf)
   else
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} ${HADOOP_CONF_DIR})
   fi
   if [ `echo ${HADOOP_HOME}/hadoop-core*.jar` != 
 ${HADOOP_HOME}/hadoop-core*.jar ] ; then
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} `ls 
 ${HADOOP_HOME}/hadoop-core*.jar | head -1`)
   else
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} `ls 
 ${HADOOP_HOME}/hadoop-common*.jar | head -1`)
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} `ls 
 ${HADOOP_HOME}/hadoop-hdfs*.jar | head -1`)
 HADOOPCPPATH=$(append_path ${HADOOPCPPATH} `ls 
 ${HADOOP_HOME}/hadoop-mapred*.jar | head -1`)
   fi
 {noformat}
 There's a couple of issues with the above code:
0. HADOOP_HOME is now deprecated in Hadoop 0.23
1. the list of jar files added to the class-path should be revised
2. we need to figure out a more robust way to get the jar files that are 
 needed to the classpath (things like hadoop-mapred*.jar tend to match 
 src/test jars as well)
 Better yet, it would be useful to look into whether we can transition HBase's 
 bin/hbase onto using bin/hadoop as a launcher script instead of direct JAVA 
 invocations (Pig, Hive, Sqoop and Mahout already do that)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5371) Introduce AccessControllerProtocol.checkPermissions(Permission[] permissons) API


[ 
https://issues.apache.org/jira/browse/HBASE-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221800#comment-13221800
 ] 

Hudson commented on HBASE-5371:
---

Integrated in HBase-TRUNK-security #128 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/128/])
HBASE-5371 Introduce AccessControllerProtocol.checkPermissions(Permission[] 
permissons) API, addendum (Enis) (Revision 1296709)

 Result = FAILURE
tedyu : 
Files : 
* 
/hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
* 
/hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControllerProtocol.java


 Introduce AccessControllerProtocol.checkPermissions(Permission[] permissons) 
 API
 

 Key: HBASE-5371
 URL: https://issues.apache.org/jira/browse/HBASE-5371
 Project: HBase
  Issue Type: Sub-task
  Components: security
Affects Versions: 0.92.1, 0.94.0
Reporter: Enis Soztutar
Assignee: Enis Soztutar
 Fix For: 0.94.0

 Attachments: HBASE-5371-addendum_v1.patch, HBASE-5371_v2.patch, 
 HBASE-5371_v3-noprefix.patch, HBASE-5371_v3.patch


 We need to introduce something like 
 AccessControllerProtocol.checkPermissions(Permission[] permissions) API, so 
 that clients can check access rights before carrying out the operations. We 
 need this kind of operation for HCATALOG-245, which introduces authorization 
 providers for hbase over hcat. We cannot use getUserPermissions() since it 
 requires ADMIN permissions on the global/table level.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5514) Compile against hadoop 0.24-SNAPSHOT

2012-03-03 Thread Mingjie Lai (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mingjie Lai updated HBASE-5514:
---

Description: 
Need to compile hbase against the latest hadoop trunk which just had NN HA 
merged in. 

1) add a hadoop 0.24 profile

2) HBASE-5480

3) HADOOP-8124 removed deprecated Syncable.sync(). It brings compile errors for 
hbase against hadoop trunk(0.24). TestHLogSplit and TestHLog still call the 
deprecated sync(). Need to replace it with hflush() so the compilation can 
pass. 

  was:HADOOP-8124 removed deprecated Syncable.sync(). It brings compile errors 
for hbase against hadoop trunk(0.24). TestHLogSplit and TestHLog still call the 
deprecated sync(). Need to replace it with hflush() so the compilation can 
pass. 

Summary: Compile against hadoop 0.24-SNAPSHOT  (was: Minor fixes to 
compile against Hadoop 0.24-SNAPSHOT: replace deprecated Syncable.sync())

@Zhihong, thanks for pointing it out. I thought hadoop hflush() was brought at 
0.20, but actually 0.21. 

As Zhihong mentioned, the scope is broader than I originally expected.

Rename the title from ``Minor fixes to compile against Hadoop 0.24-SNAPSHOT: 
replace deprecated Syncable.sync()'' - ``Compile against hadoop 
0.24-SNAPSHOT''.

 Compile against hadoop 0.24-SNAPSHOT
 

 Key: HBASE-5514
 URL: https://issues.apache.org/jira/browse/HBASE-5514
 Project: HBase
  Issue Type: Bug
  Components: build, test
Affects Versions: 0.92.0, 0.94.0
Reporter: Mingjie Lai
Assignee: Mingjie Lai
Priority: Minor
 Fix For: 0.94.0

 Attachments: HBASE-5514-2.patch, HBASE-5514.patch


 Need to compile hbase against the latest hadoop trunk which just had NN HA 
 merged in. 
 1) add a hadoop 0.24 profile
 2) HBASE-5480
 3) HADOOP-8124 removed deprecated Syncable.sync(). It brings compile errors 
 for hbase against hadoop trunk(0.24). TestHLogSplit and TestHLog still call 
 the deprecated sync(). Need to replace it with hflush() so the compilation 
 can pass. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4890) fix possible NPE in HConnectionManager

2012-03-03 Thread ramkrishna.s.vasudevan (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221801#comment-13221801
 ] 

Lars Hofhansl commented on HBASE-4890:
--

I agree, this is scary. I too will some debugging on Monday.

 fix possible NPE in HConnectionManager
 --

 Key: HBASE-4890
 URL: https://issues.apache.org/jira/browse/HBASE-4890
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jonathan Hsieh
Priority: Blocker
 Fix For: 0.92.1


 I was running YCSB against a 0.92 branch and encountered this error message:
 {code}
 11/11/29 08:47:16 WARN client.HConnectionManager$HConnectionImplementation: 
 Failed all from 
 region=usertable,user3917479014967760871,1322555655231.f78d161e5724495a9723bcd972f97f41.,
  hostname=c0316.hal.cloudera.com, port=57020
 java.util.concurrent.ExecutionException: java.lang.RuntimeException: 
 java.lang.NullPointerException
 at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
 at java.util.concurrent.FutureTask.get(FutureTask.java:83)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1501)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1353)
 at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:898)
 at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:775)
 at org.apache.hadoop.hbase.client.HTable.put(HTable.java:750)
 at com.yahoo.ycsb.db.HBaseClient.update(Unknown Source)
 at com.yahoo.ycsb.DBWrapper.update(Unknown Source)
 at com.yahoo.ycsb.workloads.CoreWorkload.doTransactionUpdate(Unknown 
 Source)
 at com.yahoo.ycsb.workloads.CoreWorkload.doTransaction(Unknown Source)
 at com.yahoo.ycsb.ClientThread.run(Unknown Source)
 Caused by: java.lang.RuntimeException: java.lang.NullPointerException
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithoutRetries(HConnectionManager.java:1315)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1327)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1325)
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:619)
 Caused by: java.lang.NullPointerException
 at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:158)
 at $Proxy4.multi(Unknown Source)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1330)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1328)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithoutRetries(HConnectionManager.java:1309)
 ... 7 more
 {code}
 It looks like the NPE is caused by server being null in the MultiRespone 
 call() method.
 {code}
  public MultiResponse call() throws IOException {
  return getRegionServerWithoutRetries(
  new ServerCallableMultiResponse(connection, tableName, null) {
public MultiResponse call() throws IOException {
  return server.multi(multi);
}
@Override
public void connect(boolean reload) throws IOException {
  server =
connection.getHRegionConnection(loc.getHostname(), 
 loc.getPort());
}
  }
  );
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5517) Region Server Coprocessor : Suggestion for change when next() call with nbRows1

2012-03-03 Thread Anoop Sam John (Created) (JIRA)

Region Server Coprocessor : Suggestion for change when next() call with nbRows1


 Key: HBASE-5517
 URL: https://issues.apache.org/jira/browse/HBASE-5517
 Project: HBase
  Issue Type: Improvement
  Components: coprocessors
Affects Versions: 0.92.0
Reporter: Anoop Sam John


Originated from the discussion under HBASE-2038 [Coprocessor based IHBase]

Currently preNext() and postNext() will be called once for a next() call into 
HRegionServer.
But if the next() is being called with nbRows1, co processor should provide a 
chance to do some operation before, after every next() calls into region as 
part of call next(int scannerId, int nbRows).

In case of usage of coprocessor with IHBase, before making any calls of next() 
into a Region, we need to make a reseek() to a row based on the index 
information.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2038) Coprocessors: Region level indexing

2012-03-03 Thread Anoop Sam John (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-2038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221804#comment-13221804
 ] 

Anoop Sam John commented on HBASE-2038:
---

Created https://issues.apache.org/jira/browse/HBASE-5517 for the co processor 
change

 Coprocessors: Region level indexing
 ---

 Key: HBASE-2038
 URL: https://issues.apache.org/jira/browse/HBASE-2038
 Project: HBase
  Issue Type: New Feature
  Components: coprocessors
Reporter: Andrew Purtell
Priority: Minor

 HBASE-2037 is a good candidate to be done as coprocessor. It also serve as a 
 good goalpost for coprocessor environment design -- there should be enough of 
 it so region level indexing can be reimplemented as a coprocessor without any 
 loss of functionality. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5510) Change in LB.randomAssignment(ListServerName servers) API

2012-03-03 Thread Anoop Sam John (Updated) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Anoop Sam John updated HBASE-5510:
--

Attachment: HBase-5510_2.patch

Change in LB.randomAssignment(ListServerName servers) API
---

[jira] [Commented] (HBASE-5510) Change in LB.randomAssignment(ListServerName servers) API

2012-03-03 Thread Anoop Sam John (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221812#comment-13221812
]

Anoop Sam John commented on HBASE-5510:
---

@Ted
Addressed your comments in the new patch.
Thanks

Change in LB.randomAssignment(ListServerName servers) API
---

[jira] [Commented] (HBASE-5510) Change in LB.randomAssignment(ListServerName servers) API

[
https://issues.apache.org/jira/browse/HBASE-5510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221815#comment-13221815
]

ramkrishna.s.vasudevan commented on HBASE-5510:
---

The patch looks good to me.
@Ted
Do you want to take a look at the updated patch?

Change in LB.randomAssignment(ListServerName servers) API
---

[jira] [Commented] (HBASE-5241) Deletes should not mask Puts that come after it.


[ 
https://issues.apache.org/jira/browse/HBASE-5241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13221817#comment-13221817
 ] 

Phabricator commented on HBASE-5241:


aaiyer has commented on the revision HBASE-5241 [jira] Deletes should not mask 
Puts that come after it..

INLINE COMMENTS
  src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java:223 
mbautin, kannan: Just thinking aloud ...

If we are able to keep track of the Rows/Rows+Col, at flush time, where we 
see that a DeleteColumn/DeleteFamily is followed by a Put/KV with a higher 
memstoreTS; we might be able to skip ahead to getNextRowOrNextColumn as 
earlier, for almost all cases except ones where there actually was a back-fill.

   Would it be possible, given the hfilev2 structure, to be able to add more 
kinds of bloom blocks to keep track of this information?


REVISION DETAIL
  https://reviews.facebook.net/D1731


 Deletes should not mask Puts that come after it.
 

 Key: HBASE-5241
 URL: https://issues.apache.org/jira/browse/HBASE-5241
 Project: HBase
  Issue Type: Improvement
Reporter: Amitanand Aiyer
Assignee: Amitanand Aiyer
 Attachments: HBASE-5241.D1731.1.patch, HBASE-5241.D1731.2.patch, 
 HBASE-5241.D1731.3.patch


 Suppose that we have a delete row, and then followed by the put. The delete 
 row
 can mask the put, unless there was a major compaction in between.
 Now that we are flushing the memstoreTS to disk, along with the KVs, we 
 should be able
 to differentiate whether or not the Put happened after the Delete and offer 
 better 
 delete semantics.
 Couldn't find a pre-existing JIRA that already discusses this, so creating 
 one.
 Seems related to https://issues.apache.org/jira/browse/HBASE-2406, but is not 
 quite the same.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5515) Add a processRow API that supports atomic multiple reads and writes on a row