date:20130731


[ 
https://issues.apache.org/jira/browse/HBASE-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13724923#comment-13724923
 ] 

Hudson commented on HBASE-8960:
---

SUCCESS: Integrated in hbase-0.95 #387 (See 
[https://builds.apache.org/job/hbase-0.95/387/])
HBASE-8960: TestDistributedLogSplitting.testLogReplayForDisablingTable fails 
sometimes - Addendum2 (jeffreyz: rev 1508712)
* 
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java


 TestDistributedLogSplitting.testLogReplayForDisablingTable fails sometimes
 --

 Key: HBASE-8960
 URL: https://issues.apache.org/jira/browse/HBASE-8960
 Project: HBase
  Issue Type: Task
  Components: test
Reporter: Jimmy Xiang
Assignee: Jeffrey Zhong
Priority: Minor
 Fix For: 0.95.2

 Attachments: hbase-8960-addendum-2.patch, hbase-8960-addendum.patch, 
 hbase-8960.patch


 http://54.241.6.143/job/HBase-0.95-Hadoop-2/org.apache.hbase$hbase-server/634/testReport/junit/org.apache.hadoop.hbase.master/TestDistributedLogSplitting/testLogReplayForDisablingTable/
 {noformat}
 java.lang.AssertionError: expected:1000 but was:0
   at org.junit.Assert.fail(Assert.java:88)
   at org.junit.Assert.failNotEquals(Assert.java:743)
   at org.junit.Assert.assertEquals(Assert.java:118)
   at org.junit.Assert.assertEquals(Assert.java:555)
   at org.junit.Assert.assertEquals(Assert.java:542)
   at 
 org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testLogReplayForDisablingTable(TestDistributedLogSplitting.java:797)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
   at 
 org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
   at 
 org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
   at 
 org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
   at 
 org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Reopened] (HBASE-9093) Hbase client API: Restore the writeToWal method


 [ 
https://issues.apache.org/jira/browse/HBASE-9093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack reopened HBASE-9093:
--


Changes?  Are we not just restoring old APIs?

 Hbase client API: Restore the writeToWal method
 ---

 Key: HBASE-9093
 URL: https://issues.apache.org/jira/browse/HBASE-9093
 Project: HBase
  Issue Type: Bug
  Components: Client, Usability
Affects Versions: 0.95.0
Reporter: Hari Shreedharan
 Fix For: 0.98.0, 0.95.2

 Attachments: HBASE-9093.patch, HBASE-9093.patch, HBASE-9093.patch


 The writeToWal is used by downstream projects like Flume to disable writes to 
 WAL to improve performance when durability is not strictly required. But 
 renaming this method to setDurability forces us to use reflection to support 
 hbase versions  95 - which in turn hits performance, as this method needs to 
 be called on every single write. I recommend adding the old method back as 
 deprecated and removing it once hbase-95/96 becomes the popular version used 
 in prod.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9093) Hbase client API: Restore the writeToWal method

2013-07-31 Thread Lars Hofhansl (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-9093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13724933#comment-13724933
 ] 

Lars Hofhansl commented on HBASE-9093:
--

I meant we cannot make forward changes. Guess my point was that 0.94 does 
support the new API, and we should use the singularity to let us make API 
incompatible changes.

So the issue in flume would be that the *older* 0.94 (and 0.92) versions cannot 
be supported without reflection.
If that is major hassle, by all means lets keep the old API around, we just 
have to weigh that against generally improving the API.


 Hbase client API: Restore the writeToWal method
 ---

 Key: HBASE-9093
 URL: https://issues.apache.org/jira/browse/HBASE-9093
 Project: HBase
  Issue Type: Bug
  Components: Client, Usability
Affects Versions: 0.95.0
Reporter: Hari Shreedharan
 Fix For: 0.98.0, 0.95.2

 Attachments: HBASE-9093.patch, HBASE-9093.patch, HBASE-9093.patch


 The writeToWal is used by downstream projects like Flume to disable writes to 
 WAL to improve performance when durability is not strictly required. But 
 renaming this method to setDurability forces us to use reflection to support 
 hbase versions  95 - which in turn hits performance, as this method needs to 
 be called on every single write. I recommend adding the old method back as 
 deprecated and removing it once hbase-95/96 becomes the popular version used 
 in prod.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9084) HBase admin flush has a data loss risk even after HBASE-7671


[ 
https://issues.apache.org/jira/browse/HBASE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13724934#comment-13724934
 ] 

chunhui shen commented on HBASE-9084:
-

Seems have such a risk.

What about using the following fix:
{code}
try {
boolean result = internalFlushcache(status);

if (coprocessorHost != null) {
  status.setStatus(Running post-flush coprocessor hooks);
  coprocessorHost.postFlush();
}

status.markComplete(Flush successful);
return result;
  }catch(DroppedSnapshotException ex){
if(rsServices!=null){
rsServices.abort(Replay of HLog required. Forcing server shutdown, ex);
}
return false;
}

{code}

I think the above code could fix the hole

 HBase admin flush has a data loss risk even after HBASE-7671
 

 Key: HBASE-9084
 URL: https://issues.apache.org/jira/browse/HBASE-9084
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.95.1, 0.94.10
Reporter: Liang Xie
Assignee: Liang Xie
Priority: Critical
 Attachments: HBASE-9084-0.94.txt, HBASE-9084.txt


 see 
 https://issues.apache.org/jira/browse/HBASE-7671?focusedCommentId=13722148page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13722148
 will attach a simple patch soon

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9084) HBase admin flush has a data loss risk even after HBASE-7671


[ 
https://issues.apache.org/jira/browse/HBASE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13724940#comment-13724940
 ] 

chunhui shen commented on HBASE-9084:
-

It means move the call abort code to HRegion, thus due to this.rsServices != 
null  this.rsServices.isAborted() is not be set  should be impossible.

Correct me if something I miss

 HBase admin flush has a data loss risk even after HBASE-7671
 

 Key: HBASE-9084
 URL: https://issues.apache.org/jira/browse/HBASE-9084
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.95.1, 0.94.10
Reporter: Liang Xie
Assignee: Liang Xie
Priority: Critical
 Attachments: HBASE-9084-0.94.txt, HBASE-9084.txt


 see 
 https://issues.apache.org/jira/browse/HBASE-7671?focusedCommentId=13722148page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13722148
 will attach a simple patch soon

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7266) [89-fb] Using pread for non-compaction read request

[
https://issues.apache.org/jira/browse/HBASE-7266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13724944#comment-13724944
]

Chao Shi commented on HBASE-7266:
-

We are experiencing this too. This happens when a large number of concurrent
scan requests on a single region. Because of the lock (on seek+read), only one
handler thread is effectively working. Our scan range is very small (just a few
adjacent rows, fits into 1 block).

[89-fb] Using pread for non-compaction read request
---

Key: HBASE-7266
URL: https://issues.apache.org/jira/browse/HBASE-7266
Project: HBase
Issue Type: Improvement
Reporter: Liyin Tang

There are 2 kinds of read operations in HBase: pread and seek+read.
Pread, positional read, is stateless and create a new connection between the
DFSClient and DataNode for each operation. While seek+read is to seek to a
specific postion and prefetch blocks from data nodes. The benefit of
seek+read is that it will cache the prefetch result but the downside is it is
stateful and needs to synchronized.
So far, both compaction and scan are using seek+read, which caused some
resource contention. So using the pread for the scan request can avoid the
resource contention. In addition, the region server is able to do the
prefetch for the scan request (HBASE-6874) so that it won't be necessary to
let the DFSClient to prefetch the data any more.
I will run through the scan benchmark (with no block cache) with verify the
performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9084) HBase admin flush has a data loss risk even after HBASE-7671

2013-07-31 Thread Liang Xie (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13724945#comment-13724945
 ] 

Liang Xie commented on HBASE-9084:
--

emm, imho, it doesn't work.  data loss means the second flush(issued by 
hbaseadmin flush side) was successful, but didn't flush its own memstore 
snapshot, just flushed the previous failed old snapshot, then the second 
flush's data was lost silently...

so i thought if just add this catch statement is not enough.


or maybe i didn't get u.

 HBase admin flush has a data loss risk even after HBASE-7671
 

 Key: HBASE-9084
 URL: https://issues.apache.org/jira/browse/HBASE-9084
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.95.1, 0.94.10
Reporter: Liang Xie
Assignee: Liang Xie
Priority: Critical
 Attachments: HBASE-9084-0.94.txt, HBASE-9084.txt


 see 
 https://issues.apache.org/jira/browse/HBASE-7671?focusedCommentId=13722148page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13722148
 will attach a simple patch soon

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-8960) TestDistributedLogSplitting.testLogReplayForDisablingTable fails sometimes


[ 
https://issues.apache.org/jira/browse/HBASE-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13724948#comment-13724948
 ] 

Hudson commented on HBASE-8960:
---

SUCCESS: Integrated in HBase-TRUNK #4323 (See 
[https://builds.apache.org/job/HBase-TRUNK/4323/])
HBASE-8960: TestDistributedLogSplitting.testLogReplayForDisablingTable fails 
sometimes - Addendum2 (jeffreyz: rev 1508711)
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java


 TestDistributedLogSplitting.testLogReplayForDisablingTable fails sometimes
 --

 Key: HBASE-8960
 URL: https://issues.apache.org/jira/browse/HBASE-8960
 Project: HBase
  Issue Type: Task
  Components: test
Reporter: Jimmy Xiang
Assignee: Jeffrey Zhong
Priority: Minor
 Fix For: 0.95.2

 Attachments: hbase-8960-addendum-2.patch, hbase-8960-addendum.patch, 
 hbase-8960.patch


 http://54.241.6.143/job/HBase-0.95-Hadoop-2/org.apache.hbase$hbase-server/634/testReport/junit/org.apache.hadoop.hbase.master/TestDistributedLogSplitting/testLogReplayForDisablingTable/
 {noformat}
 java.lang.AssertionError: expected:1000 but was:0
   at org.junit.Assert.fail(Assert.java:88)
   at org.junit.Assert.failNotEquals(Assert.java:743)
   at org.junit.Assert.assertEquals(Assert.java:118)
   at org.junit.Assert.assertEquals(Assert.java:555)
   at org.junit.Assert.assertEquals(Assert.java:542)
   at 
 org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testLogReplayForDisablingTable(TestDistributedLogSplitting.java:797)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
   at 
 org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
   at 
 org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
   at 
 org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
   at 
 org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9084) HBase admin flush has a data loss risk even after HBASE-7671


[ 
https://issues.apache.org/jira/browse/HBASE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13724949#comment-13724949
 ] 

chunhui shen commented on HBASE-9084:
-

bq.the second flush(issued by hbaseadmin flush side) was successful
It won't be successful, since we have aborted the server

 HBase admin flush has a data loss risk even after HBASE-7671
 

 Key: HBASE-9084
 URL: https://issues.apache.org/jira/browse/HBASE-9084
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.95.1, 0.94.10
Reporter: Liang Xie
Assignee: Liang Xie
Priority: Critical
 Attachments: HBASE-9084-0.94.txt, HBASE-9084.txt


 see 
 https://issues.apache.org/jira/browse/HBASE-7671?focusedCommentId=13722148page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13722148
 will attach a simple patch soon

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9084) HBase admin flush has a data loss risk even after HBASE-7671


[ 
https://issues.apache.org/jira/browse/HBASE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13724953#comment-13724953
 ] 

chunhui shen commented on HBASE-9084:
-

the second flush(issued by hbaseadmin flush side) won't work by
{code}
if (this.rsServices != null  this.rsServices.isAborted()) {
  // Don't flush when server aborting, it's unsafe
  throw new IOException(Aborting flush because server is abortted...);
}
{code}

or

{code}
synchronized (writestate) {
if (!writestate.flushing  writestate.writesEnabled) {
  this.writestate.flushing = true;
} else {
  if (LOG.isDebugEnabled()) {
LOG.debug(NOT flushing memstore for region  + this
+ , flushing= + writestate.flushing + , writesEnabled=
+ writestate.writesEnabled);
  }
  status.abort(Not flushing since 
  + (writestate.flushing ? already flushing
  : writes not enabled));
  return false;
}
  }
{code}

 HBase admin flush has a data loss risk even after HBASE-7671
 

 Key: HBASE-9084
 URL: https://issues.apache.org/jira/browse/HBASE-9084
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.95.1, 0.94.10
Reporter: Liang Xie
Assignee: Liang Xie
Priority: Critical
 Attachments: HBASE-9084-0.94.txt, HBASE-9084.txt


 see 
 https://issues.apache.org/jira/browse/HBASE-7671?focusedCommentId=13722148page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13722148
 will attach a simple patch soon

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9093) Hbase client API: Restore the writeToWal method

2013-07-31 Thread Hari Shreedharan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-9093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13724952#comment-13724952
 ] 

Hari Shreedharan commented on HBASE-9093:
-

[~lhofhansl] - Thanks for your feedback. I am not against 
removing/upgrading/updating APIs. The only thing I am requesting is to 
deprecate the old API and keep them around long enough for downstream 
projects/users to upgrade to the versions of HBase where these APIs are 
removed. 

In this case, if setWriteToWal method was deprecated in 0.94 and then removed 
in 0.96, we would have made every effort to make sure we use the new 
setDurability API by the time HBase 0.96 is released (hopefully by then, most 
users would be using at least 0.94 where both APIs would have been available). 

From the javadoc, it does not look like hbase 0.94 supported the setDurability 
API though. 
(http://hbase.apache.org/0.94/apidocs/org/apache/hadoop/hbase/client/Put.html 
- is there a newer one?)



 Hbase client API: Restore the writeToWal method
 ---

 Key: HBASE-9093
 URL: https://issues.apache.org/jira/browse/HBASE-9093
 Project: HBase
  Issue Type: Bug
  Components: Client, Usability
Affects Versions: 0.95.0
Reporter: Hari Shreedharan
 Fix For: 0.98.0, 0.95.2

 Attachments: HBASE-9093.patch, HBASE-9093.patch, HBASE-9093.patch


 The writeToWal is used by downstream projects like Flume to disable writes to 
 WAL to improve performance when durability is not strictly required. But 
 renaming this method to setDurability forces us to use reflection to support 
 hbase versions  95 - which in turn hits performance, as this method needs to 
 be called on every single write. I recommend adding the old method back as 
 deprecated and removing it once hbase-95/96 becomes the popular version used 
 in prod.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-8488) HBase transitive dependencies not being pulled in when building apps like Flume which depend on HBase


[ 
https://issues.apache.org/jira/browse/HBASE-8488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725014#comment-13725014
 ] 

stack commented on HBASE-8488:
--

Here is pointer to downstream project up on github 
https://github.com/saintstack/hbase-downstreamer


 HBase transitive dependencies not being pulled in when building apps like 
 Flume which depend on HBase
 -

 Key: HBASE-8488
 URL: https://issues.apache.org/jira/browse/HBASE-8488
 Project: HBase
  Issue Type: Bug
  Components: build
Affects Versions: 0.95.0
Reporter: Roshan Naik
Assignee: stack
Priority: Blocker
 Fix For: 0.98.0, 0.95.2

 Attachments: client.tgz


 Here is a snippet of the errors seen when building against Hbase
 {code}
 [WARNING] Invalid POM for org.apache.hbase:hbase-common:jar:0.97.0-SNAPSHOT, 
 transitive dependencies (if any) will not be available, enable debug logging 
 for more details: Some problems were encountered while processing the POMs:
 [ERROR] 'dependencyManagement.dependencies.dependency.artifactId' for 
 org.apache.hbase:${compat.module}:jar with value '${compat.module}' does not 
 match a valid id pattern. @ org.apache.hbase:hbase:0.97.0-SNAPSHOT, 
 /Users/rnaik/.m2/repository/org/apache/hbase/hbase/0.97.0-SNAPSHOT/hbase-0.97.0-SNAPSHOT.pom,
  line 982, column 21
 [ERROR] 'dependencyManagement.dependencies.dependency.artifactId' for 
 org.apache.hbase:${compat.module}:test-jar with value '${compat.module}' does 
 not match a valid id pattern. @ org.apache.hbase:hbase:0.97.0-SNAPSHOT, 
 /Users/rnaik/.m2/repository/org/apache/hbase/hbase/0.97.0-SNAPSHOT/hbase-0.97.0-SNAPSHOT.pom,
  line 987, column 21
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7634) Replication handling of changes to peer clusters is inefficient

2013-07-31 Thread Gabriel Reid (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-7634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabriel Reid updated HBASE-7634:


Attachment: HBASE-7634.v5.patch

Updated patch based on reviewboard comments from J-D

No real functional changes (other than increased configurability). Most changes 
are formatting and whitespace.

 Replication handling of changes to peer clusters is inefficient
 ---

 Key: HBASE-7634
 URL: https://issues.apache.org/jira/browse/HBASE-7634
 Project: HBase
  Issue Type: Bug
  Components: Replication
Affects Versions: 0.95.2
Reporter: Gabriel Reid
 Attachments: HBASE-7634.patch, HBASE-7634.v2.patch, 
 HBASE-7634.v3.patch, HBASE-7634.v4.patch, HBASE-7634.v5.patch


 The current handling of changes to the region servers in a replication peer 
 cluster is currently quite inefficient. The list of region servers that are 
 being replicated to is only updated if there are a large number of issues 
 encountered while replicating.
 This can cause it to take quite a while to recognize that a number of the 
 regionserver in a peer cluster are no longer available. A potentially bigger 
 problem is that if a replication peer cluster is started with a small number 
 of regionservers, and then more region servers are added after replication 
 has started, the additional region servers will never be used for replication 
 (unless there are failures on the in-use regionservers).
 Part of the current issue is that the retry code in 
 ReplicationSource#shipEdits checks a randomly-chosen replication peer 
 regionserver (in ReplicationSource#isSlaveDown) to see if it is up after a 
 replication write has failed on a different randonly-chosen replication peer. 
 If the peer is seen as not down, another randomly-chosen peer is used for 
 writing.
 A second part of the issue is that changes to the list of region servers in a 
 peer cluster are not detected at all, and are only picked up if a certain 
 number of failures have occurred when trying to ship edits.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-8224) Add '-hadoop1' or '-hadoop2' to our version string


[ 
https://issues.apache.org/jira/browse/HBASE-8224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725026#comment-13725026
 ] 

stack commented on HBASE-8224:
--

So our poms in repo, even if local repo, include the compat.module variable. It 
is not being interpolated.  Here is a build of hbase against hadoop2 profile:

{code}
durruti:hbase stack$ grep -r 'compat.module' .
./hbase/0.95.2-SNAPSHOT/hbase-0.95.2-SNAPSHOT.pom:
artifactId${compat.module}/artifactId
./hbase/0.95.2-SNAPSHOT/hbase-0.95.2-SNAPSHOT.pom:
artifactId${compat.module}/artifactId
./hbase/0.95.2-SNAPSHOT/hbase-0.95.2-SNAPSHOT.pom:
compat.modulehbase-hadoop1-compat/compat.module
./hbase/0.95.2-SNAPSHOT/hbase-0.95.2-SNAPSHOT.pom:!-- Need to set this 
for the Hadoop 1 compat module --
./hbase/0.95.2-SNAPSHOT/hbase-0.95.2-SNAPSHOT.pom:
compat.modulehbase-hadoop1-compat/compat.module
./hbase/0.95.2-SNAPSHOT/hbase-0.95.2-SNAPSHOT.pom:
compat.modulehbase-hadoop2-compat/compat.module
./hbase-assembly/0.95.2-SNAPSHOT/hbase-assembly-0.95.2-SNAPSHOT.pom:
artifactId${compat.module}/artifactId
./hbase-assembly/0.95.2-SNAPSHOT/hbase-assembly-0.95.2-SNAPSHOT.pom:
artifactId${compat.module}/artifactId
./hbase-it/0.95.2-SNAPSHOT/hbase-it-0.95.2-SNAPSHOT.pom:  
artifactId${compat.module}/artifactId
./hbase-it/0.95.2-SNAPSHOT/hbase-it-0.95.2-SNAPSHOT.pom:  
artifactId${compat.module}/artifactId
./hbase-prefix-tree/0.95.2-SNAPSHOT/hbase-prefix-tree-0.95.2-SNAPSHOT.pom:  
artifactId${compat.module}/artifactId
./hbase-server/0.95.2-SNAPSHOT/hbase-server-0.95.2-SNAPSHOT.pom:  
artifactId${compat.module}/artifactId
./hbase-server/0.95.2-SNAPSHOT/hbase-server-0.95.2-SNAPSHOT.pom:  
artifactId${compat.module}/artifactId
{code}

I've been playing w/ replacing the above variable and then building w/ new 
poms. Here is my little script:

{code}
$ hadoop=hadoop2; for i in `find . -name pom.xml`; do echo $i; 
fn=pom.xml.${hadoop}; sed 
s/\${compat.module}/hbase-${hadoop}-compat/;s/relativePath\.\./..\/relativePatch$fn/;s/\(module[^]*\)/\1\/$fn/
  $i  $i.${hadoop}; done
{code}

This puts a file beside the original pom.xml named pom.xml.hadoop2.  This file 
has a couple of substitutions done on original pom (I tried to put it into 
target dir so I did not mod the original srcs but mvn wants the modules under 
it -- and then child modules want to ref the parent -- I suppose I could make 
this work w/ more hackery).

Now building I do this:

mvn clean install -DskipTests -Dhadoop.profile=2.0 -f pom.xml.hadoop2

And repo seems better.

My downstream project is failing still though it now no longer has hadoop1 on 
refernces or fail because it wants hadoop1 dependency.

It is failing here:

{code}
  4 
---
  3 Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.67 sec 
 FAILURE!
  2 testSpinUpMiniHBaseCluster(org.hbase.downstreamer.TestHBaseMiniCluster)  
Time elapsed: 0.631 sec   ERROR!
  1 java.lang.UnsupportedOperationException: Not implemented by the 
DistributedFileSystem FileSystem implementation
...
{code}

... which uusually means I was compiled against wrong version  Looking.

 Add '-hadoop1' or '-hadoop2' to our version string
 --

 Key: HBASE-8224
 URL: https://issues.apache.org/jira/browse/HBASE-8224
 Project: HBase
  Issue Type: Task
Reporter: stack
Assignee: stack
Priority: Blocker
 Fix For: 0.95.2

 Attachments: 8224-adding.classifiers.txt, hbase-8224-proto1.patch


 So we can publish both the hadoop1 and the hadoop2 jars to a maven 
 repository, and so we can publish two packages, one for hadoop1 and one for 
 hadoop2, given how maven works, our only alternative (to the best of my 
 knowledge and after consulting others) is by amending the version string to 
 include hadoop1 or hadoop2.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-9094) Rationalize dependencies; fix analysis complaints about used but non-declared dependencies

[
https://issues.apache.org/jira/browse/HBASE-9094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-9094:
-

Attachment: dep3.txt

Some more fixup that comes of building against hadoop2. There are others still
remaining -- stuff we reference but we do not declare it in poms. Will try
again to fix these but meantime let this run against hadoopqa.

Rationalize dependencies; fix analysis complaints about used but non-declared
dependencies
--

Key: HBASE-9094
URL: https://issues.apache.org/jira/browse/HBASE-9094
Project: HBase
Issue Type: Sub-task
Components: build
Reporter: stack
Assignee: stack
Fix For: 0.98.0, 0.95.2

Attachments: dep2.txt, dep3.txt, dep.txt

Do a cleanup pass through our dependency set so downstreamers get good
picture of what they need to include looking at pom.
Poking with dependency plugin, found the following issues:
+ hbase-common is missing listing of a bunch of commons libs
+ Some of our classes make use of slf4j for no good reason. zk, thrift,
netty, and yammer, use slf4j but no need for us to be in the slf4j managing
game... so I undid our use of it and stop its transitive include everywhere.
+ hbase-examples and hbase-it do not declare includes like hbase-client,
hbase-protocol, etc.
+ hbase-hadoop1-compat depended on log4j but doesn't use it.
+ hbase-prefix-tree depends on hadoop but does not declare it (uses
WritiableUtil and RawComparator -- just two imports... uses the Audience
annotations which we could just remove but still these two critical includes)
+ hbase-server wasn't declaring its use of commons-*. Also added excludes of
transitive include of log4j by zk and yammer. Got rid of slf4j as a
dependency.
+ Add declarations for used libs such as httpclient and commons-math to
top-level pom.
+ Removed setting versions on hadoop1 and hadoop2 profile for slf4j mess.

[jira] [Updated] (HBASE-8768) Improve bulk load performance by moving key value construction from map phase to reduce phase.

2013-07-31 Thread rajeshbabu (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-8768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

rajeshbabu updated HBASE-8768:
--

Attachment: HBASE-8768_v4.patch

Patch addressing Ted's comments.

[~yuzhih...@gmail.com]
Thanks for review.
bq. Why is a new map created in each iteration of the loop
The next set will be created only after reaching configured thresold. Its not
for each iteration. Handled all other comments in the current patch.

Improve bulk load performance by moving key value construction from map phase
to reduce phase.
--

Key: HBASE-8768
URL: https://issues.apache.org/jira/browse/HBASE-8768
Project: HBase
Issue Type: Improvement
Components: mapreduce, Performance
Reporter: rajeshbabu
Assignee: rajeshbabu
Fix For: 0.98.0, 0.95.2

Attachments: HBASE-8768_v2.patch, HBASE-8768_v3.patch,
HBASE-8768_v4.patch, HBase_Bulkload_Performance_Improvement.pdf

ImportTSV bulkloading approach uses MapReduce framework. Existing mapper and
reducer classes used by ImportTSV are TsvImporterMapper.java and
PutSortReducer.java. ImportTSV tool parses the tab(by default) seperated
values from the input files and Mapper class generates the PUT objects for
each row using the Key value pairs created from the parsed text.
PutSortReducer then uses the partions based on the regions and sorts the Put
objects for each region.
Overheads we can see in the above approach:
==
1) keyvalue construction for each parsed value in the line adding extra data
like rowkey,columnfamily,qualifier which will increase around 5x extra data
to be shuffled in reduce phase.
We can calculate data size to shuffled as below
{code}
Data to be shuffled = nl*nt*(rl+cfl+cql+vall+tsl+30)
{code}
If we move keyvalue construction to reduce phase we datasize to be shuffle
will be which is very less compared to above.
{code}
Data to be shuffled = nl*nt*vall
{code}
nl - Number of lines in the raw file
nt - Number of tabs or columns including row key.
rl - row length which will be different for each line.
cfl - column family length which will be different for each family
cql - qualifier length
tsl - timestamp length.
vall- each parsed value length.
30 bytes for kv size,number of families etc.
2) In mapper side we are creating put objects by adding all keyvalues
constructed for each line and in reducer we will again collect keyvalues from
put and sort them.
Instead we can directly create and sort keyvalues in reducer.
Solution:

We can improve bulk load performance by moving the key value construction
from mapper to reducer so that Mapper just sends the raw text for each row to
the Reducer. Reducer then parses the records for rows and create and sort the
key value pairs before writing to HFiles.
Conclusion:
===
The above suggestions will improve map phase performance by avoiding keyvalue
construction and reduce phase performance by avoiding excess data to be
shuffled.

[jira] [Updated] (HBASE-8548) postOpen hook called twice

2013-07-31 Thread Nicolas Liochon (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicolas Liochon updated HBASE-8548:
---

Attachment: 8548.v1.patch

 postOpen hook called twice
 --

 Key: HBASE-8548
 URL: https://issues.apache.org/jira/browse/HBASE-8548
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.95.0
 Environment: CentOS
Reporter: Roger Ruiz-Carrillo
 Attachments: 8548.v1.patch, HRegion_HBASE-8548-0.95.patch


 postOpen hook is called twice when a region is initializing:
 Once at the end of the body of the  initializeRegionInternals() method of the 
 HRegion class.
 Once at the end initializeRegionStores() method of the HRegion class; 
 initializeRegionStores() is called inside initializeRegionInternals() and as 
 such causes the postOpen hook to be called twice.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-8548) postOpen hook called twice

2013-07-31 Thread Nicolas Liochon (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725054#comment-13725054
 ] 

Nicolas Liochon commented on HBASE-8548:


I made an error and committed the fix for this bug while committing HBASE-6295. 
 I don't know why the tests didn't pass in this jira.

So the patch here only contains a test. I've checked that the test does not 
pass with the 0.95 version from the 20th of June (git 4bfe077).

I would like to commit it on 0.95 and trunk. 


 postOpen hook called twice
 --

 Key: HBASE-8548
 URL: https://issues.apache.org/jira/browse/HBASE-8548
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.95.0
 Environment: CentOS
Reporter: Roger Ruiz-Carrillo
 Attachments: 8548.v1.patch, HRegion_HBASE-8548-0.95.patch


 postOpen hook is called twice when a region is initializing:
 Once at the end of the body of the  initializeRegionInternals() method of the 
 HRegion class.
 Once at the end initializeRegionStores() method of the HRegion class; 
 initializeRegionStores() is called inside initializeRegionInternals() and as 
 such causes the postOpen hook to be called twice.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-8548) postOpen hook called twice

2013-07-31 Thread Nicolas Liochon (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicolas Liochon updated HBASE-8548:
---

Assignee: Nicolas Liochon
Release Note:   (was: Affects only 0.95 (0.97 seems ok))
  Status: Patch Available  (was: Open)

 postOpen hook called twice
 --

 Key: HBASE-8548
 URL: https://issues.apache.org/jira/browse/HBASE-8548
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.95.0
 Environment: CentOS
Reporter: Roger Ruiz-Carrillo
Assignee: Nicolas Liochon
 Attachments: 8548.v1.patch, HRegion_HBASE-8548-0.95.patch


 postOpen hook is called twice when a region is initializing:
 Once at the end of the body of the  initializeRegionInternals() method of the 
 HRegion class.
 Once at the end initializeRegionStores() method of the HRegion class; 
 initializeRegionStores() is called inside initializeRegionInternals() and as 
 such causes the postOpen hook to be called twice.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-9101) Addendum to pluggable RpcScheduler

Chao Shi created HBASE-9101:
---

 Summary: Addendum to pluggable RpcScheduler
 Key: HBASE-9101
 URL: https://issues.apache.org/jira/browse/HBASE-9101
 Project: HBase
  Issue Type: Improvement
  Components: IPC/RPC
Reporter: Chao Shi


This patch fixes the review comments from [~stack] and a small fix:
- Make RpcScheduler fully pluggable. One can write its own implementation and 
add it to classpath and specify it by config 
hbase.region.server.rpc.scheduler.factory.class.
- Add unit tests and fix that RpcScheduler.stop is not called (discovered by 
tests)


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-9101) Addendum to pluggable RpcScheduler


 [ 
https://issues.apache.org/jira/browse/HBASE-9101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chao Shi updated HBASE-9101:


Attachment: hbase-9101.patch

try hudson

 Addendum to pluggable RpcScheduler
 --

 Key: HBASE-9101
 URL: https://issues.apache.org/jira/browse/HBASE-9101
 Project: HBase
  Issue Type: Improvement
  Components: IPC/RPC
Reporter: Chao Shi
 Attachments: hbase-9101.patch


 This patch fixes the review comments from [~stack] and a small fix:
 - Make RpcScheduler fully pluggable. One can write its own implementation and 
 add it to classpath and specify it by config 
 hbase.region.server.rpc.scheduler.factory.class.
 - Add unit tests and fix that RpcScheduler.stop is not called (discovered by 
 tests)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-9101) Addendum to pluggable RpcScheduler


 [ 
https://issues.apache.org/jira/browse/HBASE-9101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chao Shi updated HBASE-9101:


Status: Patch Available  (was: Open)

 Addendum to pluggable RpcScheduler
 --

 Key: HBASE-9101
 URL: https://issues.apache.org/jira/browse/HBASE-9101
 Project: HBase
  Issue Type: Improvement
  Components: IPC/RPC
Reporter: Chao Shi
 Attachments: hbase-9101.patch


 This patch fixes the review comments from [~stack] and a small fix:
 - Make RpcScheduler fully pluggable. One can write its own implementation and 
 add it to classpath and specify it by config 
 hbase.region.server.rpc.scheduler.factory.class.
 - Add unit tests and fix that RpcScheduler.stop is not called (discovered by 
 tests)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9101) Addendum to pluggable RpcScheduler


[ 
https://issues.apache.org/jira/browse/HBASE-9101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725072#comment-13725072
 ] 

Chao Shi commented on HBASE-9101:
-

https://reviews.apache.org/r/13106/

 Addendum to pluggable RpcScheduler
 --

 Key: HBASE-9101
 URL: https://issues.apache.org/jira/browse/HBASE-9101
 Project: HBase
  Issue Type: Improvement
  Components: IPC/RPC
Reporter: Chao Shi
 Attachments: hbase-9101.patch


 This patch fixes the review comments from [~stack] and a small fix:
 - Make RpcScheduler fully pluggable. One can write its own implementation and 
 add it to classpath and specify it by config 
 hbase.region.server.rpc.scheduler.factory.class.
 - Add unit tests and fix that RpcScheduler.stop is not called (discovered by 
 tests)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-8884) Pluggable RpcScheduler

[
https://issues.apache.org/jira/browse/HBASE-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725073#comment-13725073
]

Chao Shi commented on HBASE-8884:
-

Hi stack,

I opened another issue (HBASE-9101) and posted a patch to fix the issue you
mentioned. Please have a look. Thanks!

Pluggable RpcScheduler
--

Key: HBASE-8884
URL: https://issues.apache.org/jira/browse/HBASE-8884
Project: HBase
Issue Type: Improvement
Components: IPC/RPC
Reporter: Chao Shi
Assignee: Chao Shi
Fix For: 0.98.0

Attachments: hbase-8884.patch, hbase-8884-v2.patch,
hbase-8884-v3.patch, hbase-8884-v4.patch, hbase-8884-v5.patch,
hbase-8884-v6.patch, hbase-8884-v7.patch, hbase-8884-v8.patch

Today, the RPC scheduling mechanism is pretty simple: it execute requests in
isolated thread-pools based on their priority. In the current implementation,
all normal get/put requests are using the same pool. We'd like to add some
per-user or per-region level isolation, so that a misbehaved user/region will
not saturate the thread-pool and cause DoS to others easily. The idea is
similar to FairScheduler in MR. The current scheduling code is not standalone
and is mixed with others (Connection#processRequest). The issue is the first
step to extract it to an interface, so that people are free to write and test
their own implementations.
This patch doesn't make it completely pluggable yet, as some parameters are
pass from constructor. This is because HMaster and HRegionServer both use
RpcServer and they have different thread-pool size config. Let me know if you
have a solution to this.

[jira] [Commented] (HBASE-9094) Rationalize dependencies; fix analysis complaints about used but non-declared dependencies

2013-07-31 Thread Lars Francke (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-9094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725078#comment-13725078
]

Lars Francke commented on HBASE-9094:
-

I'd be happy to take a look at this tomorrow.

Rationalize dependencies; fix analysis complaints about used but non-declared
dependencies
--

Key: HBASE-9094
URL: https://issues.apache.org/jira/browse/HBASE-9094
Project: HBase
Issue Type: Sub-task
Components: build
Reporter: stack
Assignee: stack
Fix For: 0.98.0, 0.95.2

Attachments: dep2.txt, dep3.txt, dep.txt

[jira] [Updated] (HBASE-9090) cleanup snapshot tests setup/teardown code

2013-07-31 Thread Matteo Bertozzi (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-9090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-9090:
---

Summary: cleanup snapshot tests setup/teardown code   (was: cleanup 
snapshots setup/teardown code )

 cleanup snapshot tests setup/teardown code 
 ---

 Key: HBASE-9090
 URL: https://issues.apache.org/jira/browse/HBASE-9090
 Project: HBase
  Issue Type: Test
  Components: snapshots, test
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 0.98.0, 0.95.2

 Attachments: HBASE-9090-v0.patch, HBASE-9090-v1.patch


 There's a lot of duplicated code around createTable() and loadTable() and 
 other some stuff are done slightly different in each test (e.g. the snapshot 
 deletion in the teardown)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7634) Replication handling of changes to peer clusters is inefficient

[
https://issues.apache.org/jira/browse/HBASE-7634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725081#comment-13725081
]

Hadoop QA commented on HBASE-7634:
--

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12595145/HBASE-7634.v5.patch
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.

{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.

{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.

{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.

{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.

{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.

{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100

{color:green}+1 site{color}. The mvn site goal succeeds with this patch.

{color:green}+1 core tests{color}. The patch passed unit tests in .

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6538//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6538//console

This message is automatically generated.

Replication handling of changes to peer clusters is inefficient
---

Key: HBASE-7634
URL: https://issues.apache.org/jira/browse/HBASE-7634
Project: HBase
Issue Type: Bug
Components: Replication
Affects Versions: 0.95.2
Reporter: Gabriel Reid
Attachments: HBASE-7634.patch, HBASE-7634.v2.patch,
HBASE-7634.v3.patch, HBASE-7634.v4.patch, HBASE-7634.v5.patch

The current handling of changes to the region servers in a replication peer
cluster is currently quite inefficient. The list of region servers that are
being replicated to is only updated if there are a large number of issues
encountered while replicating.
This can cause it to take quite a while to recognize that a number of the
regionserver in a peer cluster are no longer available. A potentially bigger
problem is that if a replication peer cluster is started with a small number
of regionservers, and then more region servers are added after replication
has started, the additional region servers will never be used for replication
(unless there are failures on the in-use regionservers).
Part of the current issue is that the retry code in
ReplicationSource#shipEdits checks a randomly-chosen replication peer
regionserver (in ReplicationSource#isSlaveDown) to see if it is up after a
replication write has failed on a different randonly-chosen replication peer.
If the peer is seen as not down, another randomly-chosen peer is used for
writing.
A second part of the issue is that changes to the list of region servers in a
peer cluster are not detected at all, and are only picked up if a certain
number of failures have occurred when trying to ship edits.

[jira] [Commented] (HBASE-9094) Rationalize dependencies; fix analysis complaints about used but non-declared dependencies

[
https://issues.apache.org/jira/browse/HBASE-9094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725085#comment-13725085
]

Hadoop QA commented on HBASE-9094:
--

{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12595146/dep3.txt
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.

{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.

{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.

{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.

{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.

{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100

{color:green}+1 site{color}. The mvn site goal succeeds with this patch.

{color:green}+1 core tests{color}. The patch passed unit tests in .

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6539//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6539//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6539//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6539//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6539//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6539//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6539//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6539//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6539//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6539//console

This message is automatically generated.

Rationalize dependencies; fix analysis complaints about used but non-declared
dependencies
--

Key: HBASE-9094
URL: https://issues.apache.org/jira/browse/HBASE-9094
Project: HBase
Issue Type: Sub-task
Components: build
Reporter: stack
Assignee: stack
Fix For: 0.98.0, 0.95.2

Attachments: dep2.txt, dep3.txt, dep.txt

[jira] [Updated] (HBASE-9090) cleanup snapshot tests setup/teardown code

2013-07-31 Thread Matteo Bertozzi (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-9090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-9090:
---

   Resolution: Fixed
Fix Version/s: 0.94.11
   Status: Resolved  (was: Patch Available)

committed to 0.94, 0.95 and trunk, thanks!

 cleanup snapshot tests setup/teardown code 
 ---

 Key: HBASE-9090
 URL: https://issues.apache.org/jira/browse/HBASE-9090
 Project: HBase
  Issue Type: Test
  Components: snapshots, test
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 0.98.0, 0.95.2, 0.94.11

 Attachments: HBASE-9090-v0.patch, HBASE-9090-v1.patch


 There's a lot of duplicated code around createTable() and loadTable() and 
 other some stuff are done slightly different in each test (e.g. the snapshot 
 deletion in the teardown)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-8768) Improve bulk load performance by moving key value construction from map phase to reduce phase.


[ 
https://issues.apache.org/jira/browse/HBASE-8768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725136#comment-13725136
 ] 

Hadoop QA commented on HBASE-8768:
--

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12595148/HBASE-8768_v4.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 6 new 
or modified tests.

{color:green}+1 hadoop1.0{color}.  The patch compiles against the hadoop 
1.0 profile.

{color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoop 
2.0 profile.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6540//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6540//console

This message is automatically generated.

 Improve bulk load performance by moving key value construction from map phase 
 to reduce phase.
 --

 Key: HBASE-8768
 URL: https://issues.apache.org/jira/browse/HBASE-8768
 Project: HBase
  Issue Type: Improvement
  Components: mapreduce, Performance
Reporter: rajeshbabu
Assignee: rajeshbabu
 Fix For: 0.98.0, 0.95.2

 Attachments: HBASE-8768_v2.patch, HBASE-8768_v3.patch, 
 HBASE-8768_v4.patch, HBase_Bulkload_Performance_Improvement.pdf


 ImportTSV bulkloading approach uses MapReduce framework. Existing mapper and 
 reducer classes used by ImportTSV are TsvImporterMapper.java and 
 PutSortReducer.java. ImportTSV tool parses the tab(by default) seperated 
 values from the input files and Mapper class generates the PUT objects for 
 each row using the Key value pairs created from the parsed text. 
 PutSortReducer then uses the partions based on the regions and sorts the Put 
 objects for each region. 
 Overheads we can see in the above approach:
 ==
 1) keyvalue construction for each parsed value in the line adding extra data 
 like rowkey,columnfamily,qualifier which will increase around 5x extra data 
 to be shuffled in reduce phase.
 We can calculate data size to shuffled as below
 {code}
  Data to be shuffled = nl*nt*(rl+cfl+cql+vall+tsl+30)
 {code}
 If we move keyvalue construction to reduce phase we datasize to be shuffle 
 will be which is very less compared to above.
 {code}
  Data to be shuffled = nl*nt*vall
 {code}
 nl - Number of lines in the raw file
 nt - Number of tabs or columns including row key.
 rl - row length which will be different for each line.
 cfl - column family length which will be different for each family
 cql - qualifier length
 tsl - timestamp length.
 vall- each parsed value length.
 30 bytes for kv size,number of families etc.
 2) In mapper side we are

[jira] [Commented] (HBASE-8548) postOpen hook called twice


[ 
https://issues.apache.org/jira/browse/HBASE-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725140#comment-13725140
 ] 

Hadoop QA commented on HBASE-8548:
--

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12595151/8548.v1.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 6 new 
or modified tests.

{color:green}+1 hadoop1.0{color}.  The patch compiles against the hadoop 
1.0 profile.

{color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoop 
2.0 profile.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6541//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6541//console

This message is automatically generated.

 postOpen hook called twice
 --

 Key: HBASE-8548
 URL: https://issues.apache.org/jira/browse/HBASE-8548
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.95.0
 Environment: CentOS
Reporter: Roger Ruiz-Carrillo
Assignee: Nicolas Liochon
 Attachments: 8548.v1.patch, HRegion_HBASE-8548-0.95.patch


 postOpen hook is called twice when a region is initializing:
 Once at the end of the body of the  initializeRegionInternals() method of the 
 HRegion class.
 Once at the end initializeRegionStores() method of the HRegion class; 
 initializeRegionStores() is called inside initializeRegionInternals() and as 
 such causes the postOpen hook to be called twice.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7634) Replication handling of changes to peer clusters is inefficient

2013-07-31 Thread Gabriel Reid (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-7634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabriel Reid updated HBASE-7634:


Attachment: HBASE-7634.v6.patch

Now without javadoc warnings

 Replication handling of changes to peer clusters is inefficient
 ---

 Key: HBASE-7634
 URL: https://issues.apache.org/jira/browse/HBASE-7634
 Project: HBase
  Issue Type: Bug
  Components: Replication
Affects Versions: 0.95.2
Reporter: Gabriel Reid
 Attachments: HBASE-7634.patch, HBASE-7634.v2.patch, 
 HBASE-7634.v3.patch, HBASE-7634.v4.patch, HBASE-7634.v5.patch, 
 HBASE-7634.v6.patch


 The current handling of changes to the region servers in a replication peer 
 cluster is currently quite inefficient. The list of region servers that are 
 being replicated to is only updated if there are a large number of issues 
 encountered while replicating.
 This can cause it to take quite a while to recognize that a number of the 
 regionserver in a peer cluster are no longer available. A potentially bigger 
 problem is that if a replication peer cluster is started with a small number 
 of regionservers, and then more region servers are added after replication 
 has started, the additional region servers will never be used for replication 
 (unless there are failures on the in-use regionservers).
 Part of the current issue is that the retry code in 
 ReplicationSource#shipEdits checks a randomly-chosen replication peer 
 regionserver (in ReplicationSource#isSlaveDown) to see if it is up after a 
 replication write has failed on a different randonly-chosen replication peer. 
 If the peer is seen as not down, another randomly-chosen peer is used for 
 writing.
 A second part of the issue is that changes to the list of region servers in a 
 peer cluster are not detected at all, and are only picked up if a certain 
 number of failures have occurred when trying to ship edits.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9090) cleanup snapshot tests setup/teardown code


[ 
https://issues.apache.org/jira/browse/HBASE-9090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725147#comment-13725147
 ] 

Hudson commented on HBASE-9090:
---

SUCCESS: Integrated in hbase-0.95-on-hadoop2 #210 (See 
[https://builds.apache.org/job/hbase-0.95-on-hadoop2/210/])
HBASE-9090 cleanup snapshot tests setup/teardown code (mbertozzi: rev 1508804)
* 
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestCloneSnapshotFromClient.java
* 
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestRestoreSnapshotFromClient.java
* 
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotCloneIndependence.java
* 
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromClient.java
* 
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestSnapshotFromMaster.java
* 
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/SnapshotTestingUtils.java
* 
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
* 
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestFlushSnapshotFromClient.java
* 
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestRestoreFlushSnapshotFromClient.java


 cleanup snapshot tests setup/teardown code 
 ---

 Key: HBASE-9090
 URL: https://issues.apache.org/jira/browse/HBASE-9090
 Project: HBase
  Issue Type: Test
  Components: snapshots, test
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 0.98.0, 0.95.2, 0.94.11

 Attachments: HBASE-9090-v0.patch, HBASE-9090-v1.patch


 There's a lot of duplicated code around createTable() and loadTable() and 
 other some stuff are done slightly different in each test (e.g. the snapshot 
 deletion in the teardown)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-8960) TestDistributedLogSplitting.testLogReplayForDisablingTable fails sometimes


[ 
https://issues.apache.org/jira/browse/HBASE-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725148#comment-13725148
 ] 

Hudson commented on HBASE-8960:
---

SUCCESS: Integrated in hbase-0.95-on-hadoop2 #210 (See 
[https://builds.apache.org/job/hbase-0.95-on-hadoop2/210/])
HBASE-8960: TestDistributedLogSplitting.testLogReplayForDisablingTable fails 
sometimes - Addendum2 (jeffreyz: rev 1508712)
* 
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java


 TestDistributedLogSplitting.testLogReplayForDisablingTable fails sometimes
 --

 Key: HBASE-8960
 URL: https://issues.apache.org/jira/browse/HBASE-8960
 Project: HBase
  Issue Type: Task
  Components: test
Reporter: Jimmy Xiang
Assignee: Jeffrey Zhong
Priority: Minor
 Fix For: 0.95.2

 Attachments: hbase-8960-addendum-2.patch, hbase-8960-addendum.patch, 
 hbase-8960.patch


 http://54.241.6.143/job/HBase-0.95-Hadoop-2/org.apache.hbase$hbase-server/634/testReport/junit/org.apache.hadoop.hbase.master/TestDistributedLogSplitting/testLogReplayForDisablingTable/
 {noformat}
 java.lang.AssertionError: expected:1000 but was:0
   at org.junit.Assert.fail(Assert.java:88)
   at org.junit.Assert.failNotEquals(Assert.java:743)
   at org.junit.Assert.assertEquals(Assert.java:118)
   at org.junit.Assert.assertEquals(Assert.java:555)
   at org.junit.Assert.assertEquals(Assert.java:542)
   at 
 org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testLogReplayForDisablingTable(TestDistributedLogSplitting.java:797)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
   at 
 org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
   at 
 org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
   at 
 org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
   at 
 org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9093) Hbase client API: Restore the writeToWal method


[ 
https://issues.apache.org/jira/browse/HBASE-9093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725146#comment-13725146
 ] 

Hudson commented on HBASE-9093:
---

SUCCESS: Integrated in hbase-0.95-on-hadoop2 #210 (See 
[https://builds.apache.org/job/hbase-0.95-on-hadoop2/210/])
HBASE-9093 Hbase client API: Restore the writeToWal method (stack: rev 1508703)
* 
/hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Mutation.java
* 
/hbase/branches/0.95/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestPutWriteToWal.java


 Hbase client API: Restore the writeToWal method
 ---

 Key: HBASE-9093
 URL: https://issues.apache.org/jira/browse/HBASE-9093
 Project: HBase
  Issue Type: Bug
  Components: Client, Usability
Affects Versions: 0.95.0
Reporter: Hari Shreedharan
 Fix For: 0.98.0, 0.95.2

 Attachments: HBASE-9093.patch, HBASE-9093.patch, HBASE-9093.patch


 The writeToWal is used by downstream projects like Flume to disable writes to 
 WAL to improve performance when durability is not strictly required. But 
 renaming this method to setDurability forces us to use reflection to support 
 hbase versions  95 - which in turn hits performance, as this method needs to 
 be called on every single write. I recommend adding the old method back as 
 deprecated and removing it once hbase-95/96 becomes the popular version used 
 in prod.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9101) Addendum to pluggable RpcScheduler


[ 
https://issues.apache.org/jira/browse/HBASE-9101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725152#comment-13725152
 ] 

Hadoop QA commented on HBASE-9101:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12595152/hbase-9101.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 5 new 
or modified tests.

{color:green}+1 hadoop1.0{color}.  The patch compiles against the hadoop 
1.0 profile.

{color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoop 
2.0 profile.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces lines longer than 
100

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   org.apache.hadoop.hbase.TestCheckTestClasses

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6542//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6542//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6542//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6542//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6542//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6542//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6542//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6542//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6542//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/6542//console

This message is automatically generated.

 Addendum to pluggable RpcScheduler
 --

 Key: HBASE-9101
 URL: https://issues.apache.org/jira/browse/HBASE-9101
 Project: HBase
  Issue Type: Improvement
  Components: IPC/RPC
Reporter: Chao Shi
 Attachments: hbase-9101.patch


 This patch fixes the review comments from [~stack] and a small fix:
 - Make RpcScheduler fully pluggable. One can write its own implementation and 
 add it to classpath and specify it by config 
 hbase.region.server.rpc.scheduler.factory.class.
 - Add unit tests and fix that RpcScheduler.stop is not called (discovered by 
 tests)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-9090) cleanup snapshot tests setup/teardown code


[ 
https://issues.apache.org/jira/browse/HBASE-9090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725159#comment-13725159
 ] 

Hudson commented on HBASE-9090:
---

SUCCESS: Integrated in HBase-0.94-security #238 (See 
[https://builds.apache.org/job/HBase-0.94-security/238/])
HBASE-9090 cleanup snapshot tests setup/teardown code (mbertozzi: rev 1508811)
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestCloneSnapshotFromClient.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestRestoreSnapshotFromClient.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromClient.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestSnapshotFromMaster.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/snapshot/SnapshotTestingUtils.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/snapshot/TestFlushSnapshotFromClient.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/snapshot/TestRestoreFlushSnapshotFromClient.java


 cleanup snapshot tests setup/teardown code 
 ---

 Key: HBASE-9090
 URL: https://issues.apache.org/jira/browse/HBASE-9090
 Project: HBase
  Issue Type: Test
  Components: snapshots, test
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 0.98.0, 0.95.2, 0.94.11

 Attachments: HBASE-9090-v0.patch, HBASE-9090-v1.patch


 There's a lot of duplicated code around createTable() and loadTable() and 
 other some stuff are done slightly different in each test (e.g. the snapshot 
 deletion in the teardown)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-8768) Improve bulk load performance by moving key value construction from map phase to reduce phase.

2013-07-31 Thread Jean-Marc Spaggiari (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-8768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725174#comment-13725174
]

Jean-Marc Spaggiari commented on HBASE-8768:

Do we have any numbers regarding the improvements? Like it's 10% faster% 50%
faster?

Improve bulk load performance by moving key value construction from map phase
to reduce phase.
--

Attachments: HBASE-8768_v2.patch, HBASE-8768_v3.patch,
HBASE-8768_v4.patch, HBase_Bulkload_Performance_Improvement.pdf

[jira] [Commented] (HBASE-8768) Improve bulk load performance by moving key value construction from map phase to reduce phase.

2013-07-31 Thread rajeshbabu (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-8768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725184#comment-13725184
]

rajeshbabu commented on HBASE-8768:
---

[~jmspaggi]
bq. Do we have any numbers regarding the improvements? Like it's 10% faster%
50% faster?
performance results included in this pdf.
https://issues.apache.org/jira/secure/attachment/12594382/HBase_Bulkload_Performance_Improvement.pdf

Improve bulk load performance by moving key value construction from map phase
to reduce phase.
--

Attachments: HBASE-8768_v2.patch, HBASE-8768_v3.patch,
HBASE-8768_v4.patch, HBase_Bulkload_Performance_Improvement.pdf

[jira] [Commented] (HBASE-8768) Improve bulk load performance by moving key value construction from map phase to reduce phase.

2013-07-31 Thread Jean-Marc Spaggiari (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-8768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725186#comment-13725186
]

Jean-Marc Spaggiari commented on HBASE-8768:

Awesome! Thanks a lot [~rajesh23]

Improve bulk load performance by moving key value construction from map phase
to reduce phase.
--

Attachments: HBASE-8768_v2.patch, HBASE-8768_v3.patch,
HBASE-8768_v4.patch, HBase_Bulkload_Performance_Improvement.pdf

[jira] [Commented] (HBASE-7634) Replication handling of changes to peer clusters is inefficient