[jira] [Updated] (HBASE-16363) Length of column qualifier in Cell is a short or int
[
https://issues.apache.org/jira/browse/HBASE-16363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-16363:
-
Description:
In Cell interface (hbase-common/src/main/java/org/apache/hadoop/hbase/Cell.java)
In the comment, we have
{code}
/**
* Contiguous raw bytes that may start at any index in the containing array.
Max length is
* Short.MAX_VALUE which is 32,767 bytes.
* @return The array containing the qualifier bytes.
*/
{code}
The length of the qualifier is a short
But getQualifierLength() returns int
{code}
int getQualifierLength();
{code}
Which one is correct?
was:
In Cell interface (hbase-common/src/main/java/org/apache/hadoop/hbase/Cell.java)
In the comment, we have
{code}
/**
* Contiguous raw bytes that may start at any index in the containing array.
Max length is
* Short.MAX_VALUE which is 32,767 bytes.
* @return The array containing the qualifier bytes.
*/
{code}
The length of the qualifier is a short
But getQualifierLength() returns int
{code}
int getQualifierLength();
{code}
Which one is correct?
> Length of column qualifier in Cell is a short or int
>
>
> Key: HBASE-16363
> URL: https://issues.apache.org/jira/browse/HBASE-16363
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> In Cell interface
> (hbase-common/src/main/java/org/apache/hadoop/hbase/Cell.java)
> In the comment, we have
> {code}
> /**
> * Contiguous raw bytes that may start at any index in the containing array.
> Max length is
> * Short.MAX_VALUE which is 32,767 bytes.
> * @return The array containing the qualifier bytes.
> */
> {code}
> The length of the qualifier is a short
> But getQualifierLength() returns int
> {code}
> int getQualifierLength();
> {code}
> Which one is correct?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16285) Drop RPC requests if it must be considered as timeout at client
[
https://issues.apache.org/jira/browse/HBASE-16285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412963#comment-15412963
]
Hadoop QA commented on HBASE-16285:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
27s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s
{color} | {color:green} branch-1 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} branch-1 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s {color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 52s
{color} | {color:red} hbase-server in branch-1 has 1 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 38s
{color} | {color:green} branch-1 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} branch-1 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 43s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 84m 44s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 122m 11s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.mapreduce.TestMultiTableSnapshotInputFormat
|
| | hadoop.hbase.mapred.TestMultiTableSnapshotInputFormat |
| | hadoop.hbase.TestPartialResultsFromClientSide |
| Timed out junit tests | org.apache.hadoop.hbase.ipc.TestProtoBufRpc |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:date2016-08-09 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12822721/HBASE-16285-branch-1-v5.patch
|
| JIRA Issue | HBASE-16285 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux c063c39abcbe
[jira] [Commented] (HBASE-16377) ServerName check is ineffective in region_mover.rb
[
https://issues.apache.org/jira/browse/HBASE-16377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412954#comment-15412954
]
Ted Yu commented on HBASE-16377:
The quoted log was redacted - there was no uppercase letter originally.
So I think the cause may not be HBASE-16222.
> ServerName check is ineffective in region_mover.rb
> --
>
> Key: HBASE-16377
> URL: https://issues.apache.org/jira/browse/HBASE-16377
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 16377.branch-1.v1.txt, 16377.branch-1.v1.txt
>
>
> The following was observed during test of region_mover.rb :
> {code}
> 2016-08-08
> 11:17:05,341|beaver.machine|INFO|352|139926954637120|MainThread|2016-08-08
> 11:17:05,340 INFO [RubyThread-9: hbase-client/bin/thread-pool.rb:28]
> region_mover: Moving region hbase:meta,,1.1588230740 (1 of 14) from
> xYz.openstacklocal,16020,1470654716593, to
> server=xYz.openstacklocal,16020,1470654716593
> {code}
> There is check that target server should not be the same as current server:
> {code}
> if currentServer and currentServer == servername
> $LOG.info("Region " + r.getRegionNameAsString() + " (" + counter.to_s +
> " of " + regions.length.to_s + ") already on target server=" +
> servername)
> counter = counter + 1
> next
> end
> {code}
> However, the check is not effective.
> See comparison between object1 and object3:
> http://www.skorks.com/2009/09/ruby-equality-and-object-comparison/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16367) Race between master and region server initialization may lead to premature server abort
[
https://issues.apache.org/jira/browse/HBASE-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412951#comment-15412951
]
Ted Yu commented on HBASE-16367:
Excerpt from the attached master log:
{code}
2016-08-06 08:43:52,391 INFO [1-7:2.activeMasterManager] master.HMaster:
Initializing Master file system
...
2016-08-06 08:43:52,489 INFO [master/1-7.openstacklocal/0.0.0.18:2]
client.ZooKeeperRegistry: ClusterId read in ZooKeeper is null
2016-08-06 08:43:52,489 DEBUG [master/1-7.openstacklocal/0.0.0.18:2]
client.ConnectionManager$HConnectionImplementation: clusterid came back null,
using default default-cluster
2016-08-06 08:43:52,499 DEBUG [master/1-7.openstacklocal/0.0.0.18:2]
ipc.AbstractRpcClient:
Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4a03cc69, compressor=null,
tcpKeepAlive=true, tcpNoDelay=true, connectTO=1, readTO=2,
writeTO=6, minIdleTimeBeforeClose=12, maxRetries=0,
fallbackAllowed=false, bind address=null
2016-08-06 08:43:52,512 INFO [master/1-7.openstacklocal/0.0.0.18:2]
regionserver.HRegionServer: STOPPED: Cluster ID has not been set
{code}
HRegionServer#run() got executed before finishActiveMasterInitialization() got
to setting cluster Id.
The latch allows finishActiveMasterInitialization() to wake up
HRegionServer#run() when the cluster Id is published.
If the cluster Id is still not available after the wait, region server process
would shut down (current behavior).
Normally it shouldn't take 50 seconds for finishActiveMasterInitialization() to
publish cluster Id.
I can add a check against return value of await() being false and a debug log
so that the findbugs warning is suppressed.
> Race between master and region server initialization may lead to premature
> server abort
> ---
>
> Key: HBASE-16367
> URL: https://issues.apache.org/jira/browse/HBASE-16367
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 16367.addendum, 16367.v1.txt, 16367.v2.txt,
> 16367.v3.txt, 63908-master.log
>
>
> I was troubleshooting a case where hbase (1.1.2) master always dies shortly
> after start - see attached master log snippet.
> It turned out that master initialization thread was racing with
> HRegionServer#preRegistrationInitialization() (initializeZooKeeper, actually)
> since HMaster extends HRegionServer.
> Through additional logging in master:
> {code}
> this.oldLogDir = createInitialFileSystemLayout();
> HFileSystem.addLocationsOrderInterceptor(conf);
> LOG.info("creating splitLogManager");
> {code}
> I found that execution didn't reach the last log line before region server
> declared cluster Id being null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-9899) for idempotent operation dups, return the result instead of throwing conflict exception
[ https://issues.apache.org/jira/browse/HBASE-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412948#comment-15412948 ] Guanghao Zhang commented on HBASE-9899: --- Thanks [~stack]. But the master branch need the addendum patch too.. And it has been included in patch for branch-1. > for idempotent operation dups, return the result instead of throwing conflict > exception > --- > > Key: HBASE-9899 > URL: https://issues.apache.org/jira/browse/HBASE-9899 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Guanghao Zhang > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-9899-addendum.patch, HBASE-9899-branch-1.patch, > HBASE-9899-branch-1.patch, HBASE-9899-branch-1.patch, HBASE-9899-v1.patch, > HBASE-9899-v2.patch, HBASE-9899-v3.patch, HBASE-9899-v3.patch, > HBASE-9899-v4.patch, HBASE-9899-v4.patch > > > After HBASE-3787, we could store mvcc in operation context, and use it to > convert the modification request into read on dups instead of throwing > OperationConflictException. > MVCC tracking will have to be aware of such MVCC numbers present. Given that > scanners are usually relatively short-lived, that would prevent low watermark > from advancing for quite a bit more time -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-16377) ServerName check is ineffective in region_mover.rb
[
https://issues.apache.org/jira/browse/HBASE-16377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412945#comment-15412945
]
qgxiaozhan edited comment on HBASE-16377 at 8/9/16 5:14 AM:
I think the final reason is host name is different is Meta table between web
and zk!
see the [HBASE-16222]
was (Author: qgxiaozhan):
I think the final reason is host name is different is Meta table between web
and zk!
see the [HBASE-16222](https://issues.apache.org/jira/browse/HBASE-16222)
> ServerName check is ineffective in region_mover.rb
> --
>
> Key: HBASE-16377
> URL: https://issues.apache.org/jira/browse/HBASE-16377
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 16377.branch-1.v1.txt, 16377.branch-1.v1.txt
>
>
> The following was observed during test of region_mover.rb :
> {code}
> 2016-08-08
> 11:17:05,341|beaver.machine|INFO|352|139926954637120|MainThread|2016-08-08
> 11:17:05,340 INFO [RubyThread-9: hbase-client/bin/thread-pool.rb:28]
> region_mover: Moving region hbase:meta,,1.1588230740 (1 of 14) from
> xYz.openstacklocal,16020,1470654716593, to
> server=xYz.openstacklocal,16020,1470654716593
> {code}
> There is check that target server should not be the same as current server:
> {code}
> if currentServer and currentServer == servername
> $LOG.info("Region " + r.getRegionNameAsString() + " (" + counter.to_s +
> " of " + regions.length.to_s + ") already on target server=" +
> servername)
> counter = counter + 1
> next
> end
> {code}
> However, the check is not effective.
> See comparison between object1 and object3:
> http://www.skorks.com/2009/09/ruby-equality-and-object-comparison/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-10070) HBase read high-availability using timeline-consistent region replicas
[ https://issues.apache.org/jira/browse/HBASE-10070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-10070: -- Assignee: Enis Soztutar (was: Ashish Singhi) > HBase read high-availability using timeline-consistent region replicas > -- > > Key: HBASE-10070 > URL: https://issues.apache.org/jira/browse/HBASE-10070 > Project: HBase > Issue Type: New Feature > Components: Admin, API, LatencyResilience >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Labels: needs_releasenote > Fix For: 2.0.0, 1.2.0 > > Attachments: HighAvailabilityDesignforreadsApachedoc.pdf > > > In the present HBase architecture, it is hard, probably impossible, to > satisfy constraints like 99th percentile of the reads will be served under 10 > ms. One of the major factors that affects this is the MTTR for regions. There > are three phases in the MTTR process - detection, assignment, and recovery. > Of these, the detection is usually the longest and is presently in the order > of 20-30 seconds. During this time, the clients would not be able to read the > region data. > However, some clients will be better served if regions will be available for > reads during recovery for doing eventually consistent reads. This will help > with satisfying low latency guarantees for some class of applications which > can work with stale reads. > For improving read availability, we propose a replicated read-only region > serving design, also referred as secondary regions, or region shadows. > Extending current model of a region being opened for reads and writes in a > single region server, the region will be also opened for reading in region > servers. The region server which hosts the region for reads and writes (as in > current case) will be declared as PRIMARY, while 0 or more region servers > might be hosting the region as SECONDARY. There may be more than one > secondary (replica count > 2). > Will attach a design doc shortly which contains most of the details and some > thoughts about development approaches. Reviews are more than welcome. > We also have a proof of concept patch, which includes the master and regions > server side of changes. Client side changes will be coming soon as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-10070) HBase read high-availability using timeline-consistent region replicas
[ https://issues.apache.org/jira/browse/HBASE-10070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi reassigned HBASE-10070: - Assignee: Ashish Singhi (was: Enis Soztutar) > HBase read high-availability using timeline-consistent region replicas > -- > > Key: HBASE-10070 > URL: https://issues.apache.org/jira/browse/HBASE-10070 > Project: HBase > Issue Type: New Feature > Components: Admin, API, LatencyResilience >Reporter: Enis Soztutar >Assignee: Ashish Singhi > Labels: needs_releasenote > Fix For: 2.0.0, 1.2.0 > > Attachments: HighAvailabilityDesignforreadsApachedoc.pdf > > > In the present HBase architecture, it is hard, probably impossible, to > satisfy constraints like 99th percentile of the reads will be served under 10 > ms. One of the major factors that affects this is the MTTR for regions. There > are three phases in the MTTR process - detection, assignment, and recovery. > Of these, the detection is usually the longest and is presently in the order > of 20-30 seconds. During this time, the clients would not be able to read the > region data. > However, some clients will be better served if regions will be available for > reads during recovery for doing eventually consistent reads. This will help > with satisfying low latency guarantees for some class of applications which > can work with stale reads. > For improving read availability, we propose a replicated read-only region > serving design, also referred as secondary regions, or region shadows. > Extending current model of a region being opened for reads and writes in a > single region server, the region will be also opened for reading in region > servers. The region server which hosts the region for reads and writes (as in > current case) will be declared as PRIMARY, while 0 or more region servers > might be hosting the region as SECONDARY. There may be more than one > secondary (replica count > 2). > Will attach a design doc shortly which contains most of the details and some > thoughts about development approaches. Reviews are more than welcome. > We also have a proof of concept patch, which includes the master and regions > server side of changes. Client side changes will be coming soon as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16381) Shell deleteall command should support row key prefixes
Andrew Purtell created HBASE-16381: -- Summary: Shell deleteall command should support row key prefixes Key: HBASE-16381 URL: https://issues.apache.org/jira/browse/HBASE-16381 Project: HBase Issue Type: Improvement Components: shell Reporter: Andrew Purtell Priority: Minor The shell's deleteall command should support deleting a row range using a row key prefix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12770) Don't transfer all the queued hlogs of a dead server to the same alive server
[ https://issues.apache.org/jira/browse/HBASE-12770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412942#comment-15412942 ] Phil Yang commented on HBASE-12770: --- Thanks [~yuzhih...@gmail.com] and [~Apache9] for reviewing. > Don't transfer all the queued hlogs of a dead server to the same alive server > - > > Key: HBASE-12770 > URL: https://issues.apache.org/jira/browse/HBASE-12770 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 2.0.0, 1.4.0 >Reporter: Jianwei Cui >Assignee: Phil Yang >Priority: Minor > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-12770-branch-1-v1.patch, > HBASE-12770-branch-1-v2.patch, HBASE-12770-branch-1-v3.patch, > HBASE-12770-branch-1-v3.patch, HBASE-12770-branch-1-v3.patch, > HBASE-12770-branch-1-v3.patch, HBASE-12770-trunk.patch, HBASE-12770-v1.patch, > HBASE-12770-v2.patch, HBASE-12770-v3.patch, HBASE-12770-v3.patch > > > When a region server is down(or the cluster restart), all the hlog queues > will be transferred by the same alive region server. In a shared cluster, we > might create several peers replicating data to different peer clusters. There > might be lots of hlogs queued for these peers caused by several reasons, such > as some peers might be disabled, or errors from peer cluster might prevent > the replication, or the replication sources may fail to read some hlog > because of hdfs problem. Then, if the server is down or restarted, another > alive server will take all the replication jobs of the dead server, this > might bring a big pressure to resources(network/disk read) of the alive > server and also is not fast enough to replicate the queued hlogs. And if the > alive server is down, all the replication jobs including that takes from > other dead servers will once again be totally transferred to another alive > server, this might cause a server have a large number of queued hlogs(in our > shared cluster, we find one server might have thousands of queued hlogs for > replication). As an optional way, is it reasonable that the alive server only > transfer one peer's hlogs from the dead server one time? Then, other alive > region servers might have the opportunity to transfer the hlogs of rest > peers. This may also help the queued hlogs be processed more fast. Any > discussion is welcome. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16367) Race between master and region server initialization may lead to premature server abort
[
https://issues.apache.org/jira/browse/HBASE-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412933#comment-15412933
]
stack commented on HBASE-16367:
---
What is this? It adds:
821 if (this.initLatch != null) {
822 this.initLatch.await(50, TimeUnit.SECONDS);
823 }
...which causes a new findbugs reported above but ignored:
Return value of java.util.concurrent.CountDownLatch.await(long, TimeUnit)
ignored in
org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper() At
HRegionServer.java:ignored in
org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper() At
HRegionServer.java:[line 822]
We then wait on the latch 50 seconds and then just proceed? What is supposed to
be the startup scenario here? How does the latch ensure a particular path?
> Race between master and region server initialization may lead to premature
> server abort
> ---
>
> Key: HBASE-16367
> URL: https://issues.apache.org/jira/browse/HBASE-16367
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 16367.addendum, 16367.v1.txt, 16367.v2.txt,
> 16367.v3.txt, 63908-master.log
>
>
> I was troubleshooting a case where hbase (1.1.2) master always dies shortly
> after start - see attached master log snippet.
> It turned out that master initialization thread was racing with
> HRegionServer#preRegistrationInitialization() (initializeZooKeeper, actually)
> since HMaster extends HRegionServer.
> Through additional logging in master:
> {code}
> this.oldLogDir = createInitialFileSystemLayout();
> HFileSystem.addLocationsOrderInterceptor(conf);
> LOG.info("creating splitLogManager");
> {code}
> I found that execution didn't reach the last log line before region server
> declared cluster Id being null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16299) Update REST API scanner with ability to do reverse scan
[ https://issues.apache.org/jira/browse/HBASE-16299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412932#comment-15412932 ] ramkrishna.s.vasudevan commented on HBASE-16299: [~minwoo.kang] Since the patches are committed can you close the RB requests that you had opened. > Update REST API scanner with ability to do reverse scan > --- > > Key: HBASE-16299 > URL: https://issues.apache.org/jira/browse/HBASE-16299 > Project: HBase > Issue Type: Improvement > Components: REST >Affects Versions: 1.1.2 > Environment: Not environment specific (tested on HDP 2.4.2) >Reporter: Bjorn Olsen >Assignee: Minwoo Kang >Priority: Minor > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-16299.branch-1.3.001.patch, > HBASE-16299.master.001.patch > > > HBASE-4811 "Support reverse scan" was implemented from version 0.98.0, and is > available in the Java API. > However this functionality is not yet exposed via REST. > Example of expected API call: > http://server:port/table/*?startrow=1=10=true; > (Returns rows ordered by key in reverse, eg from 9*** to 1*** ) > Based on my (limited) understanding this should be simple to add. > See org.apach.hadoop.hbase.rest.TableResource.getScanResource > This function creates a Scan object with parameters passed in from the REST > API (I assume). > Adding this functionality may be as simple as adding a "reversed" parameter > to the REST API which passes down to where the Scan object is created in > TableResource.getScanResource. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15554) StoreFile$Writer.appendGeneralBloomFilter generates extra KV
[ https://issues.apache.org/jira/browse/HBASE-15554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15554: --- Attachment: HBASE-15554_11.patch The last patch added one new java doc and whitespace issue when I applied the eclipse based formatting short cut. All tests have already passed with the last patch. > StoreFile$Writer.appendGeneralBloomFilter generates extra KV > > > Key: HBASE-15554 > URL: https://issues.apache.org/jira/browse/HBASE-15554 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: Vladimir Rodionov >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > Attachments: HBASE-15554.patch, HBASE-15554_10.patch, > HBASE-15554_11.patch, HBASE-15554_3.patch, HBASE-15554_4.patch, > HBASE-15554_6.patch, HBASE-15554_7.patch, HBASE-15554_9.patch > > > Accounts for 10% memory allocation in compaction thread when BloomFilterType > is ROWCOL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15554) StoreFile$Writer.appendGeneralBloomFilter generates extra KV
[ https://issues.apache.org/jira/browse/HBASE-15554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15554: --- Status: Patch Available (was: Open) > StoreFile$Writer.appendGeneralBloomFilter generates extra KV > > > Key: HBASE-15554 > URL: https://issues.apache.org/jira/browse/HBASE-15554 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: Vladimir Rodionov >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > Attachments: HBASE-15554.patch, HBASE-15554_10.patch, > HBASE-15554_11.patch, HBASE-15554_3.patch, HBASE-15554_4.patch, > HBASE-15554_6.patch, HBASE-15554_7.patch, HBASE-15554_9.patch > > > Accounts for 10% memory allocation in compaction thread when BloomFilterType > is ROWCOL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15554) StoreFile$Writer.appendGeneralBloomFilter generates extra KV
[ https://issues.apache.org/jira/browse/HBASE-15554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15554: --- Status: Open (was: Patch Available) > StoreFile$Writer.appendGeneralBloomFilter generates extra KV > > > Key: HBASE-15554 > URL: https://issues.apache.org/jira/browse/HBASE-15554 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: Vladimir Rodionov >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > Attachments: HBASE-15554.patch, HBASE-15554_10.patch, > HBASE-15554_3.patch, HBASE-15554_4.patch, HBASE-15554_6.patch, > HBASE-15554_7.patch, HBASE-15554_9.patch > > > Accounts for 10% memory allocation in compaction thread when BloomFilterType > is ROWCOL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16308) Contain protobuf references
[ https://issues.apache.org/jira/browse/HBASE-16308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412925#comment-15412925 ] stack commented on HBASE-16308: --- Fix the findbugs (not mine) and a NPE. There are still some weird failures in here. Digging but get a run in in the meantime. > Contain protobuf references > --- > > Key: HBASE-16308 > URL: https://issues.apache.org/jira/browse/HBASE-16308 > Project: HBase > Issue Type: Sub-task > Components: Protobufs >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-16308.master.001.patch, > HBASE-16308.master.002.patch, HBASE-16308.master.003.patch, > HBASE-16308.master.004.patch, HBASE-16308.master.005.patch, > HBASE-16308.master.006.patch, HBASE-16308.master.006.patch, > HBASE-16308.master.007.patch, HBASE-16308.master.008.patch, > HBASE-16308.master.009.patch > > > Clean up our protobuf references so contained to just a few classes rather > than being spread about the codebase. Doing this work will make it easier > landing the parent issue and will make it more clear where the division > between shaded protobuf and unshaded protobuf lies (we need to continue with > unshaded protobuf for HDFS references by AsyncWAL and probably EndPoint > Coprocessors) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-9899) for idempotent operation dups, return the result instead of throwing conflict exception
[ https://issues.apache.org/jira/browse/HBASE-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9899: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 1.4.0 1.3.0 Status: Resolved (was: Patch Available) Pushed to branch-1 and branch-1.3 ([~mantonov] FYI). Thanks for the very nice patch [~zghaobac] > for idempotent operation dups, return the result instead of throwing conflict > exception > --- > > Key: HBASE-9899 > URL: https://issues.apache.org/jira/browse/HBASE-9899 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Guanghao Zhang > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-9899-addendum.patch, HBASE-9899-branch-1.patch, > HBASE-9899-branch-1.patch, HBASE-9899-branch-1.patch, HBASE-9899-v1.patch, > HBASE-9899-v2.patch, HBASE-9899-v3.patch, HBASE-9899-v3.patch, > HBASE-9899-v4.patch, HBASE-9899-v4.patch > > > After HBASE-3787, we could store mvcc in operation context, and use it to > convert the modification request into read on dups instead of throwing > OperationConflictException. > MVCC tracking will have to be aware of such MVCC numbers present. Given that > scanners are usually relatively short-lived, that would prevent low watermark > from advancing for quite a bit more time -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-16361) Revert of HBASE-16317, "revert all ESAPI..." broke TestLogLevel
[ https://issues.apache.org/jira/browse/HBASE-16361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-16361. --- Resolution: Fixed Fix Version/s: 2.0.0 Resolving as fixed by HBASE-16267. I don't see TestLogLevel failures anymore. > Revert of HBASE-16317, "revert all ESAPI..." broke TestLogLevel > --- > > Key: HBASE-16361 > URL: https://issues.apache.org/jira/browse/HBASE-16361 > Project: HBase > Issue Type: Bug > Components: dependencies, UI >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > > Side effect of the revert in HBASE-16317 is broken TestLogLevel. > testDynamicLogLevel(org.apache.hadoop.hbase.http.log.TestLogLevel) Time > elapsed: 0.956 sec <<< ERROR! > java.io.IOException: Server returned HTTP response code: 500 for URL: > http://localhost:51940/logLevel?log=org.apache.hadoop.hbase.http.log.TestLogLevel=ERROR > at > sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) > at > sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) > at > org.apache.hadoop.hbase.http.log.TestLogLevel.testDynamicLogLevel(TestLogLevel.java:71) > Complaint is because... > 41 Caused by: java.lang.ClassNotFoundException: > org.apache.commons.httpclient.URIException > 42 at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > Let me see if I can fix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16308) Contain protobuf references
[ https://issues.apache.org/jira/browse/HBASE-16308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16308: -- Attachment: HBASE-16308.master.009.patch > Contain protobuf references > --- > > Key: HBASE-16308 > URL: https://issues.apache.org/jira/browse/HBASE-16308 > Project: HBase > Issue Type: Sub-task > Components: Protobufs >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-16308.master.001.patch, > HBASE-16308.master.002.patch, HBASE-16308.master.003.patch, > HBASE-16308.master.004.patch, HBASE-16308.master.005.patch, > HBASE-16308.master.006.patch, HBASE-16308.master.006.patch, > HBASE-16308.master.007.patch, HBASE-16308.master.008.patch, > HBASE-16308.master.009.patch > > > Clean up our protobuf references so contained to just a few classes rather > than being spread about the codebase. Doing this work will make it easier > landing the parent issue and will make it more clear where the division > between shaded protobuf and unshaded protobuf lies (we need to continue with > unshaded protobuf for HDFS references by AsyncWAL and probably EndPoint > Coprocessors) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16318) fail build if license isn't in whitelist
[ https://issues.apache.org/jira/browse/HBASE-16318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-16318: Attachment: HBASE-16318.3.patch -03 - adds org.htrace from pre-incubator htrace for Hadoop 2.6.z > fail build if license isn't in whitelist > > > Key: HBASE-16318 > URL: https://issues.apache.org/jira/browse/HBASE-16318 > Project: HBase > Issue Type: Sub-task > Components: build, dependencies >Reporter: Sean Busbey >Assignee: Sean Busbey > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 1.2.3, 0.98.22 > > Attachments: HBASE-16318.0.patch, HBASE-16318.1.patch, > HBASE-16318.2.patch, HBASE-16318.3.patch > > > we use supplemental-models.xml to make sure we have consistent names and > descriptions for licenses. we also know what licenses we expect to see in our > build. If we see a different one > # fail the velocity template process > # if possible, include some information about why this happened -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16181) Allow snapshot of hbase:backup table
[ https://issues.apache.org/jira/browse/HBASE-16181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-16181: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) > Allow snapshot of hbase:backup table > > > Key: HBASE-16181 > URL: https://issues.apache.org/jira/browse/HBASE-16181 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Attachments: HBASE-16181-v1.patch, HBASE-16181-v2.patch > > > Snapshot of HBase system tables is not supported, we need either move > hbase:backup into different name space or fix snapshots. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16378) Procedure v2 - Make ProcedureException extend HBaseException
[
https://issues.apache.org/jira/browse/HBASE-16378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412910#comment-15412910
]
Hadoop QA commented on HBASE-16378:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 18s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
25s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 30s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 30s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
32s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 30s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 30s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 32s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
22s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 6s
{color} | {color:red} hbase-common generated 1 new + 0 unchanged - 0 fixed = 1
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 42s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 52s
{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 48m 28s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-common |
| | Exception is caught when Exception is not thrown in
[jira] [Updated] (HBASE-16181) Allow snapshot of hbase:backup table
[ https://issues.apache.org/jira/browse/HBASE-16181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-16181: --- Labels: backup (was: ) > Allow snapshot of hbase:backup table > > > Key: HBASE-16181 > URL: https://issues.apache.org/jira/browse/HBASE-16181 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Attachments: HBASE-16181-v1.patch, HBASE-16181-v2.patch > > > Snapshot of HBase system tables is not supported, we need either move > hbase:backup into different name space or fix snapshots. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16380) clean up 0.94.y references
[ https://issues.apache.org/jira/browse/HBASE-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-16380: Summary: clean up 0.94.y references (was: clean up 0.94 references) > clean up 0.94.y references > -- > > Key: HBASE-16380 > URL: https://issues.apache.org/jira/browse/HBASE-16380 > Project: HBase > Issue Type: Task > Components: community, website >Reporter: Sean Busbey > > consensus on [the DISCUSS thread seemed to be EOM for > 0.94|https://lists.apache.org/thread.html/547058fc59cb48130b35c80f86d41c28038195c99f5ed94e834e291c@%3Cdev.hbase.apache.org%3E] > * announce on user@ > * remove 0.94.y from dist.apache > * remove it from the ref guide > * pare down unreleased 0.94.y versions from JIRA (save at most 1) > * archive released 0.94.y versions in JIRA > * disable / delete builds.a.o jobs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16215) clean up 1.0.z references
[ https://issues.apache.org/jira/browse/HBASE-16215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412897#comment-15412897 ] Sean Busbey commented on HBASE-16215: - * disable/delete builds.a.o jobs. > clean up 1.0.z references > - > > Key: HBASE-16215 > URL: https://issues.apache.org/jira/browse/HBASE-16215 > Project: HBase > Issue Type: Task > Components: community, website >Reporter: Sean Busbey > > I've seen us state a few places that 1.0.z is EOM. We should clean up > remaining references > * remove 1.0.z artifact from dist.apache > * remove it from the ref guide (java prereqs, hadoop prereqs, RM list) > * remove branch-1.0 from git > * remove unreleased 1.0.z versions from JIRA > * archive released 1.0.z versions in JIRA -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16380) clean up 0.94 references
[ https://issues.apache.org/jira/browse/HBASE-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412896#comment-15412896 ] Sean Busbey commented on HBASE-16380: - update our dist area's HEADER.html to mark it EOM as well > clean up 0.94 references > > > Key: HBASE-16380 > URL: https://issues.apache.org/jira/browse/HBASE-16380 > Project: HBase > Issue Type: Task > Components: community, website >Reporter: Sean Busbey > > consensus on [the DISCUSS thread seemed to be EOM for > 0.94|https://lists.apache.org/thread.html/547058fc59cb48130b35c80f86d41c28038195c99f5ed94e834e291c@%3Cdev.hbase.apache.org%3E] > * announce on user@ > * remove 0.94.y from dist.apache > * remove it from the ref guide > * pare down unreleased 0.94.y versions from JIRA (save at most 1) > * archive released 0.94.y versions in JIRA > * disable / delete builds.a.o jobs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16181) Allow snapshot of hbase:backup table
[ https://issues.apache.org/jira/browse/HBASE-16181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-16181: --- Summary: Allow snapshot of hbase:backup table (was: Backup of hbase:backup table) > Allow snapshot of hbase:backup table > > > Key: HBASE-16181 > URL: https://issues.apache.org/jira/browse/HBASE-16181 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-16181-v1.patch, HBASE-16181-v2.patch > > > Snapshot of HBase system tables is not supported, we need either move > hbase:backup into different name space or fix snapshots. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-16380) clean up 0.94 references
[ https://issues.apache.org/jira/browse/HBASE-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412895#comment-15412895 ] Sean Busbey edited comment on HBASE-16380 at 8/9/16 4:13 AM: - * Pending discussion mentioned on HBASE-16215, tag state of 0.94 branch(es) and remove branches * Remove from version specific docs from navigation on website (actual docs too?) was (Author: busbey): * Pending discussion mentioned on HBASE-16215, tag state of 0.94 branch(es) and remove branches * Remove from version specific docs from website > clean up 0.94 references > > > Key: HBASE-16380 > URL: https://issues.apache.org/jira/browse/HBASE-16380 > Project: HBase > Issue Type: Task > Components: community, website >Reporter: Sean Busbey > > consensus on [the DISCUSS thread seemed to be EOM for > 0.94|https://lists.apache.org/thread.html/547058fc59cb48130b35c80f86d41c28038195c99f5ed94e834e291c@%3Cdev.hbase.apache.org%3E] > * announce on user@ > * remove 0.94.y from dist.apache > * remove it from the ref guide > * pare down unreleased 0.94.y versions from JIRA (save at most 1) > * archive released 0.94.y versions in JIRA > * disable / delete builds.a.o jobs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16380) clean up 0.94 references
Sean Busbey created HBASE-16380: --- Summary: clean up 0.94 references Key: HBASE-16380 URL: https://issues.apache.org/jira/browse/HBASE-16380 Project: HBase Issue Type: Task Components: community, website Reporter: Sean Busbey consensus on [the DISCUSS thread seemed to be EOM for 0.94|https://lists.apache.org/thread.html/547058fc59cb48130b35c80f86d41c28038195c99f5ed94e834e291c@%3Cdev.hbase.apache.org%3E] * announce on user@ * remove 0.94.y from dist.apache * remove it from the ref guide * pare down unreleased 0.94.y versions from JIRA (save at most 1) * archive released 0.94.y versions in JIRA * disable / delete builds.a.o jobs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16380) clean up 0.94 references
[ https://issues.apache.org/jira/browse/HBASE-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412895#comment-15412895 ] Sean Busbey commented on HBASE-16380: - * Pending discussion mentioned on HBASE-16215, tag state of 0.94 branch(es) and remove branches * Remove from version specific docs from website > clean up 0.94 references > > > Key: HBASE-16380 > URL: https://issues.apache.org/jira/browse/HBASE-16380 > Project: HBase > Issue Type: Task > Components: community, website >Reporter: Sean Busbey > > consensus on [the DISCUSS thread seemed to be EOM for > 0.94|https://lists.apache.org/thread.html/547058fc59cb48130b35c80f86d41c28038195c99f5ed94e834e291c@%3Cdev.hbase.apache.org%3E] > * announce on user@ > * remove 0.94.y from dist.apache > * remove it from the ref guide > * pare down unreleased 0.94.y versions from JIRA (save at most 1) > * archive released 0.94.y versions in JIRA > * disable / delete builds.a.o jobs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-8731) Use the JDK 1.7 in the precommit env for trunk
[ https://issues.apache.org/jira/browse/HBASE-8731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey resolved HBASE-8731. Resolution: Fixed marking this fixed again. we've used jdk7 on master for some time. > Use the JDK 1.7 in the precommit env for trunk > -- > > Key: HBASE-8731 > URL: https://issues.apache.org/jira/browse/HBASE-8731 > Project: HBase > Issue Type: Improvement > Components: build >Affects Versions: 0.99.0 >Reporter: Nicolas Liochon >Assignee: Giridharan Kesavan > > HBase today uses the jdk 1.6. In the past it created issues when we tried to > use 1.7 for the core build while the precommit was on 1.6. > Having the precommit on 1.7 would solve this. > The best is to start with trunk. Likely 0.95 will come next, and may be, a > day, 0.94. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-9465) Push entries to peer clusters serially
[ https://issues.apache.org/jira/browse/HBASE-9465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-9465: - Attachment: HBASE-9465-branch-1-v4.patch > Push entries to peer clusters serially > -- > > Key: HBASE-9465 > URL: https://issues.apache.org/jira/browse/HBASE-9465 > Project: HBase > Issue Type: New Feature > Components: regionserver, Replication >Reporter: Honghua Feng >Assignee: Phil Yang > Attachments: HBASE-9465-branch-1-v1.patch, > HBASE-9465-branch-1-v1.patch, HBASE-9465-branch-1-v2.patch, > HBASE-9465-branch-1-v3.patch, HBASE-9465-branch-1-v4.patch, > HBASE-9465-branch-1-v4.patch, HBASE-9465-v1.patch, HBASE-9465-v2.patch, > HBASE-9465-v2.patch, HBASE-9465-v3.patch, HBASE-9465-v4.patch, > HBASE-9465-v5.patch, HBASE-9465-v6.patch, HBASE-9465-v6.patch, > HBASE-9465-v7.patch, HBASE-9465-v7.patch, HBASE-9465.pdf > > > When region-move or RS failure occurs in master cluster, the hlog entries > that are not pushed before region-move or RS-failure will be pushed by > original RS(for region move) or another RS which takes over the remained hlog > of dead RS(for RS failure), and the new entries for the same region(s) will > be pushed by the RS which now serves the region(s), but they push the hlog > entries of a same region concurrently without coordination. > This treatment can possibly lead to data inconsistency between master and > peer clusters: > 1. there are put and then delete written to master cluster > 2. due to region-move / RS-failure, they are pushed by different > replication-source threads to peer cluster > 3. if delete is pushed to peer cluster before put, and flush and > major-compact occurs in peer cluster before put is pushed to peer cluster, > the delete is collected and the put remains in peer cluster > In this scenario, the put remains in peer cluster, but in master cluster the > put is masked by the delete, hence data inconsistency between master and peer > clusters -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16285) Drop RPC requests if it must be considered as timeout at client
[ https://issues.apache.org/jira/browse/HBASE-16285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412864#comment-15412864 ] Phil Yang commented on HBASE-16285: --- Failed test on master is unrelated and can pass locally > Drop RPC requests if it must be considered as timeout at client > --- > > Key: HBASE-16285 > URL: https://issues.apache.org/jira/browse/HBASE-16285 > Project: HBase > Issue Type: Improvement >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: HBASE-16285-branch-1-v1.patch, > HBASE-16285-branch-1-v2.patch, HBASE-16285-branch-1-v3.patch, > HBASE-16285-branch-1-v4.patch, HBASE-16285-branch-1-v5.patch, > HBASE-16285-branch-1-v5.patch, HBASE-16285-v1.patch, HBASE-16285-v2.patch, > HBASE-16285-v3.patch, HBASE-16285-v4.patch, HBASE-16285-v5.patch, > HBASE-16285-v6.patch, HBASE-16285-v7.patch, HBASE-16285-v7.patch, > HBASE-16285-v8.patch > > > After HBASE-15593, we have a timeout param in header of RPC requests. We can > use it in more scenes. > A straightforward scene is to drop requests if it has waited so long in RPC > queue and has been dropped by client. Even if we handle this request and send > the response back, it will not be used any more. And client may have sent a > retry. In an extreme case, if the server is slow, all requests may be timeout > or queue-full-exception because we should handle previous requests which have > been dropped by client and many resources at server are wasted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16378) Procedure v2 - Make ProcedureException extend HBaseException
[ https://issues.apache.org/jira/browse/HBASE-16378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-16378: Status: Patch Available (was: Open) > Procedure v2 - Make ProcedureException extend HBaseException > > > Key: HBASE-16378 > URL: https://issues.apache.org/jira/browse/HBASE-16378 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 1.2.2, 1.1.5, 2.0.0, 1.3.0 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0, 1.3.0, 1.1.6, 1.2.3 > > Attachments: HBASE-16378-v0.patch > > > Make ProcedureException extend HBaseException, so we can avoid stuff like > HBASE-15591. and avoid try/catch ProcedureException and direct rethrows -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16285) Drop RPC requests if it must be considered as timeout at client
[ https://issues.apache.org/jira/browse/HBASE-16285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-16285: -- Attachment: HBASE-16285-branch-1-v5.patch Retry for branch-1 > Drop RPC requests if it must be considered as timeout at client > --- > > Key: HBASE-16285 > URL: https://issues.apache.org/jira/browse/HBASE-16285 > Project: HBase > Issue Type: Improvement >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: HBASE-16285-branch-1-v1.patch, > HBASE-16285-branch-1-v2.patch, HBASE-16285-branch-1-v3.patch, > HBASE-16285-branch-1-v4.patch, HBASE-16285-branch-1-v5.patch, > HBASE-16285-branch-1-v5.patch, HBASE-16285-v1.patch, HBASE-16285-v2.patch, > HBASE-16285-v3.patch, HBASE-16285-v4.patch, HBASE-16285-v5.patch, > HBASE-16285-v6.patch, HBASE-16285-v7.patch, HBASE-16285-v7.patch, > HBASE-16285-v8.patch > > > After HBASE-15593, we have a timeout param in header of RPC requests. We can > use it in more scenes. > A straightforward scene is to drop requests if it has waited so long in RPC > queue and has been dropped by client. Even if we handle this request and send > the response back, it will not be used any more. And client may have sent a > retry. In an extreme case, if the server is slow, all requests may be timeout > or queue-full-exception because we should handle previous requests which have > been dropped by client and many resources at server are wasted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-9465) Push entries to peer clusters serially
[ https://issues.apache.org/jira/browse/HBASE-9465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412858#comment-15412858 ] Phil Yang edited comment on HBASE-9465 at 8/9/16 3:20 AM: -- Can not see the failure message in Test Results, and it can pass locally, let's resubmit the patch to retry was (Author: yangzhe1991): Can not seen the failure message in Test Results, and it can pass locally, let's resubmit the patch to retry > Push entries to peer clusters serially > -- > > Key: HBASE-9465 > URL: https://issues.apache.org/jira/browse/HBASE-9465 > Project: HBase > Issue Type: New Feature > Components: regionserver, Replication >Reporter: Honghua Feng >Assignee: Phil Yang > Attachments: HBASE-9465-branch-1-v1.patch, > HBASE-9465-branch-1-v1.patch, HBASE-9465-branch-1-v2.patch, > HBASE-9465-branch-1-v3.patch, HBASE-9465-branch-1-v4.patch, > HBASE-9465-v1.patch, HBASE-9465-v2.patch, HBASE-9465-v2.patch, > HBASE-9465-v3.patch, HBASE-9465-v4.patch, HBASE-9465-v5.patch, > HBASE-9465-v6.patch, HBASE-9465-v6.patch, HBASE-9465-v7.patch, > HBASE-9465-v7.patch, HBASE-9465.pdf > > > When region-move or RS failure occurs in master cluster, the hlog entries > that are not pushed before region-move or RS-failure will be pushed by > original RS(for region move) or another RS which takes over the remained hlog > of dead RS(for RS failure), and the new entries for the same region(s) will > be pushed by the RS which now serves the region(s), but they push the hlog > entries of a same region concurrently without coordination. > This treatment can possibly lead to data inconsistency between master and > peer clusters: > 1. there are put and then delete written to master cluster > 2. due to region-move / RS-failure, they are pushed by different > replication-source threads to peer cluster > 3. if delete is pushed to peer cluster before put, and flush and > major-compact occurs in peer cluster before put is pushed to peer cluster, > the delete is collected and the put remains in peer cluster > In this scenario, the put remains in peer cluster, but in master cluster the > put is masked by the delete, hence data inconsistency between master and peer > clusters -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-9465) Push entries to peer clusters serially
[ https://issues.apache.org/jira/browse/HBASE-9465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412858#comment-15412858 ] Phil Yang commented on HBASE-9465: -- Can not seen the failure message in Test Results, and it can pass locally, let's resubmit the patch to retry > Push entries to peer clusters serially > -- > > Key: HBASE-9465 > URL: https://issues.apache.org/jira/browse/HBASE-9465 > Project: HBase > Issue Type: New Feature > Components: regionserver, Replication >Reporter: Honghua Feng >Assignee: Phil Yang > Attachments: HBASE-9465-branch-1-v1.patch, > HBASE-9465-branch-1-v1.patch, HBASE-9465-branch-1-v2.patch, > HBASE-9465-branch-1-v3.patch, HBASE-9465-branch-1-v4.patch, > HBASE-9465-v1.patch, HBASE-9465-v2.patch, HBASE-9465-v2.patch, > HBASE-9465-v3.patch, HBASE-9465-v4.patch, HBASE-9465-v5.patch, > HBASE-9465-v6.patch, HBASE-9465-v6.patch, HBASE-9465-v7.patch, > HBASE-9465-v7.patch, HBASE-9465.pdf > > > When region-move or RS failure occurs in master cluster, the hlog entries > that are not pushed before region-move or RS-failure will be pushed by > original RS(for region move) or another RS which takes over the remained hlog > of dead RS(for RS failure), and the new entries for the same region(s) will > be pushed by the RS which now serves the region(s), but they push the hlog > entries of a same region concurrently without coordination. > This treatment can possibly lead to data inconsistency between master and > peer clusters: > 1. there are put and then delete written to master cluster > 2. due to region-move / RS-failure, they are pushed by different > replication-source threads to peer cluster > 3. if delete is pushed to peer cluster before put, and flush and > major-compact occurs in peer cluster before put is pushed to peer cluster, > the delete is collected and the put remains in peer cluster > In this scenario, the put remains in peer cluster, but in master cluster the > put is masked by the delete, hence data inconsistency between master and peer > clusters -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-9465) Push entries to peer clusters serially
[ https://issues.apache.org/jira/browse/HBASE-9465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412855#comment-15412855 ] Phil Yang commented on HBASE-9465: -- Seems this result is tests of master, but the output is not correct... In https://builds.apache.org/job/PreCommit-HBASE-Build/3001/consoleFull we can see at start the patch is HBASE-9465-v7.patch... > Push entries to peer clusters serially > -- > > Key: HBASE-9465 > URL: https://issues.apache.org/jira/browse/HBASE-9465 > Project: HBase > Issue Type: New Feature > Components: regionserver, Replication >Reporter: Honghua Feng >Assignee: Phil Yang > Attachments: HBASE-9465-branch-1-v1.patch, > HBASE-9465-branch-1-v1.patch, HBASE-9465-branch-1-v2.patch, > HBASE-9465-branch-1-v3.patch, HBASE-9465-branch-1-v4.patch, > HBASE-9465-v1.patch, HBASE-9465-v2.patch, HBASE-9465-v2.patch, > HBASE-9465-v3.patch, HBASE-9465-v4.patch, HBASE-9465-v5.patch, > HBASE-9465-v6.patch, HBASE-9465-v6.patch, HBASE-9465-v7.patch, > HBASE-9465-v7.patch, HBASE-9465.pdf > > > When region-move or RS failure occurs in master cluster, the hlog entries > that are not pushed before region-move or RS-failure will be pushed by > original RS(for region move) or another RS which takes over the remained hlog > of dead RS(for RS failure), and the new entries for the same region(s) will > be pushed by the RS which now serves the region(s), but they push the hlog > entries of a same region concurrently without coordination. > This treatment can possibly lead to data inconsistency between master and > peer clusters: > 1. there are put and then delete written to master cluster > 2. due to region-move / RS-failure, they are pushed by different > replication-source threads to peer cluster > 3. if delete is pushed to peer cluster before put, and flush and > major-compact occurs in peer cluster before put is pushed to peer cluster, > the delete is collected and the put remains in peer cluster > In this scenario, the put remains in peer cluster, but in master cluster the > put is masked by the delete, hence data inconsistency between master and peer > clusters -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-9465) Push entries to peer clusters serially
[ https://issues.apache.org/jira/browse/HBASE-9465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-9465: - Attachment: HBASE-9465-v7.patch Retry for master > Push entries to peer clusters serially > -- > > Key: HBASE-9465 > URL: https://issues.apache.org/jira/browse/HBASE-9465 > Project: HBase > Issue Type: New Feature > Components: regionserver, Replication >Reporter: Honghua Feng >Assignee: Phil Yang > Attachments: HBASE-9465-branch-1-v1.patch, > HBASE-9465-branch-1-v1.patch, HBASE-9465-branch-1-v2.patch, > HBASE-9465-branch-1-v3.patch, HBASE-9465-branch-1-v4.patch, > HBASE-9465-v1.patch, HBASE-9465-v2.patch, HBASE-9465-v2.patch, > HBASE-9465-v3.patch, HBASE-9465-v4.patch, HBASE-9465-v5.patch, > HBASE-9465-v6.patch, HBASE-9465-v6.patch, HBASE-9465-v7.patch, > HBASE-9465-v7.patch, HBASE-9465.pdf > > > When region-move or RS failure occurs in master cluster, the hlog entries > that are not pushed before region-move or RS-failure will be pushed by > original RS(for region move) or another RS which takes over the remained hlog > of dead RS(for RS failure), and the new entries for the same region(s) will > be pushed by the RS which now serves the region(s), but they push the hlog > entries of a same region concurrently without coordination. > This treatment can possibly lead to data inconsistency between master and > peer clusters: > 1. there are put and then delete written to master cluster > 2. due to region-move / RS-failure, they are pushed by different > replication-source threads to peer cluster > 3. if delete is pushed to peer cluster before put, and flush and > major-compact occurs in peer cluster before put is pushed to peer cluster, > the delete is collected and the put remains in peer cluster > In this scenario, the put remains in peer cluster, but in master cluster the > put is masked by the delete, hence data inconsistency between master and peer > clusters -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12721) Create Docker container cluster infrastructure to enable better testing
[
https://issues.apache.org/jira/browse/HBASE-12721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412788#comment-15412788
]
Dima Spivak commented on HBASE-12721:
-
Assuming you're running Docker 1.11+, to build HBase images from Git SHA
(defaults to using Hadoop 2.7.2):
{noformat}
source /dev/stdin <<< "$(curl -sL http://tiny.cloudera.com/clusterdock.sh)"
CLUSTERDOCK_TOPOLOGY_IMAGE=dimaspivak/clusterdock:apache_hbase_topology
clusterdock_run ./bin/build_cluster --namespace=dimaspivak apache_hbase
--hadoop-version=2.7.1
--hadoop-tarball=https://archive.apache.org/dist/hadoop/core/hadoop-2.7.1/hadoop-2.7.1.tar.gz
--hbase-version=andysCommit --hbase-git-commit
1ecb0fce342ee878cf96f7a3165007192bedb2ef
{noformat}
To start the cluster:
{noformat}
CLUSTERDOCK_TOPOLOGY_IMAGE=dimaspivak/clusterdock:apache_hbase_topology
clusterdock_run ./bin/start_cluster --namespace=dimaspivak apache_hbase
--hadoop-version=2.7.1 --hbase-version=andysCommit
--secondary-nodes='node-{2..5}'
{noformat}
To run ITBLL against the cluster:
{noformat}
clusterdock_ssh node-1.cluster
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList...
{noformat}
> Create Docker container cluster infrastructure to enable better testing
> ---
>
> Key: HBASE-12721
> URL: https://issues.apache.org/jira/browse/HBASE-12721
> Project: HBase
> Issue Type: New Feature
> Components: build, community, documentation, test
>Reporter: Dima Spivak
>Assignee: Dima Spivak
>
> Some simple work on using HBase with Docker was committed into /dev-support
> as "hbase_docker;" all this did was stand up a standalone cluster from source
> and start a shell. Now seems like a good time to extend this to be useful for
> applications that could actual benefit the community, especially around
> testing. Some ideas:
> - Integration testing would be much more accessible if people could stand up
> distributed HBase clusters on a single host machine in a couple minutes and
> run our awesome hbase-it suite against it.
> - Binary compatibility testing of an HBase client is easiest when standing up
> an HBase cluster can be done once and then different client source/binary
> permutations run against it.
> - Upgrade testing, and especially rolling upgrade testing, doesn't have any
> upstream automation on build.apache.org, in part because it's a pain to set
> up x-node clusters on Apache infrastructure.
> This proposal, whether it stays under /dev-support or moves out into it's own
> top-level module ("hbase-docker" would conveniently fit the existing schema
> :-)), strives to create a simple framework for deploying "distributed,"
> multi-container Apache HBase clusters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-16255) Backup/Restore IT
[ https://issues.apache.org/jira/browse/HBASE-16255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412743#comment-15412743 ] Vladimir Rodionov edited comment on HBASE-16255 at 8/9/16 1:13 AM: --- v2 patch. Extends IntegrationTestBase, but Monkey is disabled. We still work on Fault tolerance support (HBASE-15227), besides this there is no real IT for snapshot (nobody has ever tested snapshots with fault injections), hence we will be failing real IT (with chaos monkey enabled) because we depend on snapshot functionality. If the purpose to document only different types of failures, I can enable CM, otherwise I would postpone this (but not merge to master) until Phase 3 complete. What do you think, [~dimaspivak]? was (Author: vrodionov): v2 patch. Extends IntegrationTestBase, but Monkey is disabled. We still work on Fault tolerance support (HBASE-15227), besides this there is no real IT for snapshot (nobody has ever tested snapshots with fault injections), hence we will be failing real IT (with chaos monkey enabled) because we depends on snapshot functionality. If the purpose to document only different types of failures, I can enable CM, otherwise I would postpone this (but not merge to master) until Phase 3 complete. What do you think, [~dimaspivak]? > Backup/Restore IT > - > > Key: HBASE-16255 > URL: https://issues.apache.org/jira/browse/HBASE-16255 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Attachments: HBASE-16255-v1.patch, HBASE-16255-v2.patch > > > Integration test for backup restore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-16255) Backup/Restore IT
[ https://issues.apache.org/jira/browse/HBASE-16255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412743#comment-15412743 ] Vladimir Rodionov edited comment on HBASE-16255 at 8/9/16 1:13 AM: --- v2 patch. Extends IntegrationTestBase, but Monkey is disabled. We still work on Fault tolerance support (HBASE-15227), besides this there is no real IT for snapshot (nobody has ever tested snapshots with fault injections), hence we will be failing real IT (with chaos monkey enabled) because we depends on snapshot functionality. If the purpose to document only different types of failures, I can enable CM, otherwise I would postpone this (but not merge to master) until Phase 3 complete. What do you think, [~dimaspivak]? was (Author: vrodionov): v2 patch. Extends IntegrationTestBase, but Monkey is disabled. > Backup/Restore IT > - > > Key: HBASE-16255 > URL: https://issues.apache.org/jira/browse/HBASE-16255 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Attachments: HBASE-16255-v1.patch, HBASE-16255-v2.patch > > > Integration test for backup restore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16318) fail build if license isn't in whitelist
[
https://issues.apache.org/jira/browse/HBASE-16318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412773#comment-15412773
]
Hadoop QA commented on HBASE-16318:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 14m 10s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 7m 25s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
34s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 57s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 12s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 32s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 52s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 50s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 50s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 8s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m 8s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
32s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 20m 53s
{color} | {color:red} The patch causes 11 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 23m 45s
{color} | {color:red} The patch causes 11 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 26m 42s
{color} | {color:red} The patch causes 11 errors with Hadoop v2.6.3. {color} |
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m 13s
{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 0s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 34s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 8s
{color} | {color:green} hbase-resource-bundle in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 116m 33s
{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
53s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 211m 30s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:date2016-08-08 |
| JIRA Patch URL |
[jira] [Commented] (HBASE-16308) Contain protobuf references
[
https://issues.apache.org/jira/browse/HBASE-16308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412753#comment-15412753
]
Hadoop QA commented on HBASE-16308:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 13m 31s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 11 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 3m 48s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
23s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 58s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 17s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
46s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
43s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 2s
{color} | {color:red} hbase-server in master has 1 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 28s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 27s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
32s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 56s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 56s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 16s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 35s {color}
| {color:red} hbase-server-jdk1.7.0_101 with JDK v1.7.0_101 generated 2 new + 4
unchanged - 2 fixed = 6 total (was 6) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
55s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 2s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
58s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 14s
{color} | {color:red} hbase-client generated 1 new + 0 unchanged - 0 fixed = 1
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 7s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 26s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 42s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 15s {color}
| {color:red} hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 24m 56s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 8s
{color} | {color:green} hbase-spark in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
42s {color} | {color:green} The patch does not generate ASF License
[jira] [Commented] (HBASE-12721) Create Docker container cluster infrastructure to enable better testing
[
https://issues.apache.org/jira/browse/HBASE-12721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412752#comment-15412752
]
Dima Spivak commented on HBASE-12721:
-
New review up on RB. Update removes the bulk of the framework from Apache HBase
but leaves behind what's needed to create an {{apache_hbase}} topology that can
be plugged into the framework. Since the original patches went up,
{{clusterdock.sh}} from the framework has also been updated to make it easier
to SSH into nodes. In short, now instead of needing to worry about private
keys, assuming you've started up a cluster with nodes
{{node-\{1..5\}.cluster}}, you can simply run
{noformat}
clusterdock_ssh node-1.cluster
{noformat}
to get an interactive terminal or
{noformat}
clusterdock_ssh node-1.cluster 'echo "list" | hbase shell -n'
{noformat}
to run a command against a running Docker-based cluster.
> Create Docker container cluster infrastructure to enable better testing
> ---
>
> Key: HBASE-12721
> URL: https://issues.apache.org/jira/browse/HBASE-12721
> Project: HBase
> Issue Type: New Feature
> Components: build, community, documentation, test
>Reporter: Dima Spivak
>Assignee: Dima Spivak
>
> Some simple work on using HBase with Docker was committed into /dev-support
> as "hbase_docker;" all this did was stand up a standalone cluster from source
> and start a shell. Now seems like a good time to extend this to be useful for
> applications that could actual benefit the community, especially around
> testing. Some ideas:
> - Integration testing would be much more accessible if people could stand up
> distributed HBase clusters on a single host machine in a couple minutes and
> run our awesome hbase-it suite against it.
> - Binary compatibility testing of an HBase client is easiest when standing up
> an HBase cluster can be done once and then different client source/binary
> permutations run against it.
> - Upgrade testing, and especially rolling upgrade testing, doesn't have any
> upstream automation on build.apache.org, in part because it's a pain to set
> up x-node clusters on Apache infrastructure.
> This proposal, whether it stays under /dev-support or moves out into it's own
> top-level module ("hbase-docker" would conveniently fit the existing schema
> :-)), strives to create a simple framework for deploying "distributed,"
> multi-container Apache HBase clusters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16255) Backup/Restore IT
[ https://issues.apache.org/jira/browse/HBASE-16255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-16255: -- Attachment: HBASE-16255-v2.patch v2 patch. Extends IntegrationTestBase, but Monkey is disabled. > Backup/Restore IT > - > > Key: HBASE-16255 > URL: https://issues.apache.org/jira/browse/HBASE-16255 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Attachments: HBASE-16255-v1.patch, HBASE-16255-v2.patch > > > Integration test for backup restore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16238) It's useless to catch SESSIONEXPIRED exception and retry in RecoverableZooKeeper
[
https://issues.apache.org/jira/browse/HBASE-16238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412715#comment-15412715
]
Andrew Purtell commented on HBASE-16238:
Reverted from 0.98. This change causes TestZooKeeper to go zombie. FIx version
was set to 0.98.20 but it didn't go into that release, this would have gone out
in .21.
> It's useless to catch SESSIONEXPIRED exception and retry in
> RecoverableZooKeeper
>
>
> Key: HBASE-16238
> URL: https://issues.apache.org/jira/browse/HBASE-16238
> Project: HBase
> Issue Type: Bug
> Components: Zookeeper
>Affects Versions: 1.1.5, 1.2.2, 0.98.20
>Reporter: Allan Yang
>Priority: Minor
> Fix For: 1.3.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-16238.patch, HBASE-16238v2.patch
>
>
> After HBASE-5549, SESSIONEXPIRED exception was caught and retried with other
> zookeeper exceptions like ConnectionLoss. But it is useless to retry when a
> session expired happens, since the retry will never be successful. Though
> there is a config called "zookeeper.recovery.retry" to control retry times,
> in our cases, we set this config to a very big number like "9". When a
> session expired happens, the regionserver should kill itself, but because of
> the retrying, threads of regionserver stuck at trying to reconnect to
> zookeeper, and never properly shut down.
> {code}
> public Stat exists(String path, boolean watch)
> throws KeeperException, InterruptedException {
> TraceScope traceScope = null;
> try {
> traceScope = Trace.startSpan("RecoverableZookeeper.exists");
> RetryCounter retryCounter = retryCounterFactory.create();
> while (true) {
> try {
> return checkZk().exists(path, watch);
> } catch (KeeperException e) {
> switch (e.code()) {
> case CONNECTIONLOSS:
> case SESSIONEXPIRED: //we shouldn't catch this
> case OPERATIONTIMEOUT:
> retryOrThrow(retryCounter, e, "exists");
> break;
> default:
> throw e;
> }
> }
> retryCounter.sleepUntilNextRetry();
> }
> } finally {
> if (traceScope != null) traceScope.close();
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16238) It's useless to catch SESSIONEXPIRED exception and retry in RecoverableZooKeeper
[
https://issues.apache.org/jira/browse/HBASE-16238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-16238:
---
Fix Version/s: (was: 0.98.20)
> It's useless to catch SESSIONEXPIRED exception and retry in
> RecoverableZooKeeper
>
>
> Key: HBASE-16238
> URL: https://issues.apache.org/jira/browse/HBASE-16238
> Project: HBase
> Issue Type: Bug
> Components: Zookeeper
>Affects Versions: 1.1.5, 1.2.2, 0.98.20
>Reporter: Allan Yang
>Priority: Minor
> Fix For: 1.3.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-16238.patch, HBASE-16238v2.patch
>
>
> After HBASE-5549, SESSIONEXPIRED exception was caught and retried with other
> zookeeper exceptions like ConnectionLoss. But it is useless to retry when a
> session expired happens, since the retry will never be successful. Though
> there is a config called "zookeeper.recovery.retry" to control retry times,
> in our cases, we set this config to a very big number like "9". When a
> session expired happens, the regionserver should kill itself, but because of
> the retrying, threads of regionserver stuck at trying to reconnect to
> zookeeper, and never properly shut down.
> {code}
> public Stat exists(String path, boolean watch)
> throws KeeperException, InterruptedException {
> TraceScope traceScope = null;
> try {
> traceScope = Trace.startSpan("RecoverableZookeeper.exists");
> RetryCounter retryCounter = retryCounterFactory.create();
> while (true) {
> try {
> return checkZk().exists(path, watch);
> } catch (KeeperException e) {
> switch (e.code()) {
> case CONNECTIONLOSS:
> case SESSIONEXPIRED: //we shouldn't catch this
> case OPERATIONTIMEOUT:
> retryOrThrow(retryCounter, e, "exists");
> break;
> default:
> throw e;
> }
> }
> retryCounter.sleepUntilNextRetry();
> }
> } finally {
> if (traceScope != null) traceScope.close();
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16379) [replication] Minor improvement to replication/copy_tables_desc.rb
[
https://issues.apache.org/jira/browse/HBASE-16379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412712#comment-15412712
]
Hadoop QA commented on HBASE-16379:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 4m 50s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m 5s
{color} | {color:red} The patch generated 17 new + 15 unchanged - 5 fixed = 32
total (was 20) {color} |
| {color:red}-1{color} | {color:red} ruby-lint {color} | {color:red} 0m 1s
{color} | {color:red} The patch generated 2 new + 29 unchanged - 2 fixed = 31
total (was 31) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
42m 45s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 5m 25s
{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 2m
19s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 58m 45s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.0 Server=1.12.0 Image:yetus/hbase:date2016-08-08 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12822690/0001-HBASE-16379-replication-Minor-improvement-to-replica.patch
|
| JIRA Issue | HBASE-16379 |
| Optional Tests | asflicense rubocop ruby_lint |
| uname | Linux 36ce2492443a 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 1ecb0fc |
| rubocop | v0.42.0 |
| rubocop |
https://builds.apache.org/job/PreCommit-HBASE-Build/3016/artifact/patchprocess/diff-patch-rubocop.txt
|
| ruby-lint | v2.3.0 |
| ruby-lint |
https://builds.apache.org/job/PreCommit-HBASE-Build/3016/artifact/patchprocess/diff-patch-ruby-lint.txt
|
| hbaseprotoc |
https://builds.apache.org/job/PreCommit-HBASE-Build/3016/artifact/patchprocess/patch-hbaseprotoc-root.txt
|
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3016/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> [replication] Minor improvement to replication/copy_tables_desc.rb
> --
>
> Key: HBASE-16379
> URL: https://issues.apache.org/jira/browse/HBASE-16379
> Project: HBase
> Issue Type: Improvement
> Components: Replication, shell
>Affects Versions: 1.3.0, 1.2.2
>Reporter: Esteban Gutierrez
>Assignee: Esteban Gutierrez
>Priority: Trivial
> Attachments:
> 0001-HBASE-16379-replication-Minor-improvement-to-replica.patch
>
>
> copy_tables_desc.rb is helpful for quickly setting up a table remotely based
> on an existing schema. However it does copy by default all tables. Now you
> can pass a list of tables as an optional third argument and it will also
> display what table descriptors where copied.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16144) Replication queue's lock will live forever if RS acquiring the lock has died prematurely
[ https://issues.apache.org/jira/browse/HBASE-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-16144: --- Fix Version/s: (was: 0.98.21) I had to revert this from 0.98 because it lead to repeatable failures of TestZooKeeper#testLogSplittingAfterMasterRecoveryDueToZKExpiry Thanks [~ted_yu] for finding the problem > Replication queue's lock will live forever if RS acquiring the lock has died > prematurely > > > Key: HBASE-16144 > URL: https://issues.apache.org/jira/browse/HBASE-16144 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6 >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 1.2.3 > > Attachments: HBASE-16144-0.98.v1.patch, > HBASE-16144-branch-1-v1.patch, HBASE-16144-branch-1-v2.patch, > HBASE-16144-branch-1.1-v1.patch, HBASE-16144-branch-1.1-v2.patch, > HBASE-16144-v1.patch, HBASE-16144-v2.patch, HBASE-16144-v3.patch, > HBASE-16144-v4.patch, HBASE-16144-v5.patch, HBASE-16144-v6.patch, > HBASE-16144-v6.patch > > > In default, we will use multi operation when we claimQueues from ZK. But if > we set hbase.zookeeper.useMulti=false, we will add a lock first, then copy > nodes, finally clean old queue and the lock. > However, if the RS acquiring the lock crash before claimQueues done, the lock > will always be there and other RS can never claim the queue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16318) fail build if license isn't in whitelist
[
https://issues.apache.org/jira/browse/HBASE-16318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412697#comment-15412697
]
Hadoop QA commented on HBASE-16318:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
59s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 21s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 40s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 50s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 29s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 26s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 41s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 18m 21s
{color} | {color:red} The patch causes 11 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 20m 57s
{color} | {color:red} The patch causes 11 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 23m 25s
{color} | {color:red} The patch causes 11 errors with Hadoop v2.6.3. {color} |
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m 10s
{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 56s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 30s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 7s
{color} | {color:green} hbase-resource-bundle in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 103m 31s
{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 159m 23s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush |
| Timed out junit tests |
[jira] [Updated] (HBASE-16144) Replication queue's lock will live forever if RS acquiring the lock has died prematurely
[ https://issues.apache.org/jira/browse/HBASE-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-16144: --- Fix Version/s: 0.98.21 > Replication queue's lock will live forever if RS acquiring the lock has died > prematurely > > > Key: HBASE-16144 > URL: https://issues.apache.org/jira/browse/HBASE-16144 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6 >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 0.98.21, 1.2.3 > > Attachments: HBASE-16144-0.98.v1.patch, > HBASE-16144-branch-1-v1.patch, HBASE-16144-branch-1-v2.patch, > HBASE-16144-branch-1.1-v1.patch, HBASE-16144-branch-1.1-v2.patch, > HBASE-16144-v1.patch, HBASE-16144-v2.patch, HBASE-16144-v3.patch, > HBASE-16144-v4.patch, HBASE-16144-v5.patch, HBASE-16144-v6.patch, > HBASE-16144-v6.patch > > > In default, we will use multi operation when we claimQueues from ZK. But if > we set hbase.zookeeper.useMulti=false, we will add a lock first, then copy > nodes, finally clean old queue and the lock. > However, if the RS acquiring the lock crash before claimQueues done, the lock > will always be there and other RS can never claim the queue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16144) Replication queue's lock will live forever if RS acquiring the lock has died prematurely
[ https://issues.apache.org/jira/browse/HBASE-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-16144: --- Fix Version/s: (was: 0.98.21) > Replication queue's lock will live forever if RS acquiring the lock has died > prematurely > > > Key: HBASE-16144 > URL: https://issues.apache.org/jira/browse/HBASE-16144 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6 >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 1.2.3 > > Attachments: HBASE-16144-0.98.v1.patch, > HBASE-16144-branch-1-v1.patch, HBASE-16144-branch-1-v2.patch, > HBASE-16144-branch-1.1-v1.patch, HBASE-16144-branch-1.1-v2.patch, > HBASE-16144-v1.patch, HBASE-16144-v2.patch, HBASE-16144-v3.patch, > HBASE-16144-v4.patch, HBASE-16144-v5.patch, HBASE-16144-v6.patch, > HBASE-16144-v6.patch > > > In default, we will use multi operation when we claimQueues from ZK. But if > we set hbase.zookeeper.useMulti=false, we will add a lock first, then copy > nodes, finally clean old queue and the lock. > However, if the RS acquiring the lock crash before claimQueues done, the lock > will always be there and other RS can never claim the queue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14379) Replication V2
[ https://issues.apache.org/jira/browse/HBASE-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez updated HBASE-14379: -- Assignee: (was: Esteban Gutierrez) > Replication V2 > -- > > Key: HBASE-14379 > URL: https://issues.apache.org/jira/browse/HBASE-14379 > Project: HBase > Issue Type: Umbrella > Components: Replication >Reporter: Andrew Purtell > Fix For: 2.0.0 > > > Replication V2 is a tear-down of exiting replication code to just the > interfaces introduced in HBASE-11367, then a rebuild around the following > principles, goals, and suggested features: > - No state in ZooKeeper. Introduce a new system table for tracking peers, > queues, and log positions. (Some discussion on HBASE-10295, probably will be > replaced with a set of more focused issues.) > - Allow replication v1 and v2 to coexist. Note all of the undesirable > features of v1 will remain as long as v1 is active, 'fixing' v1 is out of > scope. Supporting communication between v1 and v2 endpoints would also be out > of scope. > - Simplified internal programming model based on iterators > - Streaming data transfer > - Administrative actions mediated by the master with support for security > hooks (like HBASE-11392) > - Replication state persisted and communicated with protobuf (like > HBASE-11393 but everywhere) > - Detailed metrics > - Support for at least simple status checks and admin actions via UI and shell > - Hbck support for fixing corrupt or stuck queues (like HBASE-14014) > - Support for bulk load, perhaps through augmenting bulk load to build WALs > as well as HFiles (see HBASE-13153) > - Optional consideration for replicating schema as well as data (like > HBASE-12947). May fall out of scope. > - Optional separation of replication function from the regionservers (see > HBASE-8772) > - Optional alternate scheduling of edits besides FIFO-by-region (see > HBASE-1734 and HBASE-14014) > There are a number of existing JIRAs that will eventually be closed as > duplicate, wont fix, or reparented here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-14379) Replication V2
[ https://issues.apache.org/jira/browse/HBASE-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez reassigned HBASE-14379: - Assignee: Esteban Gutierrez > Replication V2 > -- > > Key: HBASE-14379 > URL: https://issues.apache.org/jira/browse/HBASE-14379 > Project: HBase > Issue Type: Umbrella > Components: Replication >Reporter: Andrew Purtell >Assignee: Esteban Gutierrez > Fix For: 2.0.0 > > > Replication V2 is a tear-down of exiting replication code to just the > interfaces introduced in HBASE-11367, then a rebuild around the following > principles, goals, and suggested features: > - No state in ZooKeeper. Introduce a new system table for tracking peers, > queues, and log positions. (Some discussion on HBASE-10295, probably will be > replaced with a set of more focused issues.) > - Allow replication v1 and v2 to coexist. Note all of the undesirable > features of v1 will remain as long as v1 is active, 'fixing' v1 is out of > scope. Supporting communication between v1 and v2 endpoints would also be out > of scope. > - Simplified internal programming model based on iterators > - Streaming data transfer > - Administrative actions mediated by the master with support for security > hooks (like HBASE-11392) > - Replication state persisted and communicated with protobuf (like > HBASE-11393 but everywhere) > - Detailed metrics > - Support for at least simple status checks and admin actions via UI and shell > - Hbck support for fixing corrupt or stuck queues (like HBASE-14014) > - Support for bulk load, perhaps through augmenting bulk load to build WALs > as well as HFiles (see HBASE-13153) > - Optional consideration for replicating schema as well as data (like > HBASE-12947). May fall out of scope. > - Optional separation of replication function from the regionservers (see > HBASE-8772) > - Optional alternate scheduling of edits besides FIFO-by-region (see > HBASE-1734 and HBASE-14014) > There are a number of existing JIRAs that will eventually be closed as > duplicate, wont fix, or reparented here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16367) Race between master and region server initialization may lead to premature server abort
[
https://issues.apache.org/jira/browse/HBASE-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412651#comment-15412651
]
Hudson commented on HBASE-16367:
FAILURE: Integrated in HBase-Trunk_matrix #1379 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/1379/])
HBASE-16367 Race between master and region server initialization may (tedyu:
rev 1ecb0fce342ee878cf96f7a3165007192bedb2ef)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
> Race between master and region server initialization may lead to premature
> server abort
> ---
>
> Key: HBASE-16367
> URL: https://issues.apache.org/jira/browse/HBASE-16367
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 16367.addendum, 16367.v1.txt, 16367.v2.txt,
> 16367.v3.txt, 63908-master.log
>
>
> I was troubleshooting a case where hbase (1.1.2) master always dies shortly
> after start - see attached master log snippet.
> It turned out that master initialization thread was racing with
> HRegionServer#preRegistrationInitialization() (initializeZooKeeper, actually)
> since HMaster extends HRegionServer.
> Through additional logging in master:
> {code}
> this.oldLogDir = createInitialFileSystemLayout();
> HFileSystem.addLocationsOrderInterceptor(conf);
> LOG.info("creating splitLogManager");
> {code}
> I found that execution didn't reach the last log line before region server
> declared cluster Id being null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12947) Replicating DDL statements like create from one cluster to another
[
https://issues.apache.org/jira/browse/HBASE-12947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412646#comment-15412646
]
Esteban Gutierrez commented on HBASE-12947:
---
I ran across this recently and I concur with [~apurtell] on the the challenges.
For instance, sometimes you would like to have the tables in the sink cluster
using a different compression codec or a larger TTL than the source or
depending on the namespace the users on the remote cluster might have some
restriction on the use of coprocessors. I think tooling like
{{bin/replication/copy_tables_desc.rb}} is ok for basic management, but for
fine grain control we should revisit this as part of HBASE-14379.
> Replicating DDL statements like create from one cluster to another
> ---
>
> Key: HBASE-12947
> URL: https://issues.apache.org/jira/browse/HBASE-12947
> Project: HBase
> Issue Type: New Feature
> Components: Replication
>Affects Versions: 2.0.0
>Reporter: Prabhu Joseph
>Priority: Critical
> Fix For: 2.0.0
>
>
> Problem:
> When tables are created dynamically in Hbase cluster, the Replication
> feature can't be used as the new table does not exist in peer cluster. To use
> the replication, we need to create same table in peer cluster also.
>Having API to replicate the create table statement at peer cluster will be
> more helpful in such cases.
> Solution:
> create 'table','cf',replication => true , peerFlag => true
> if peerFlag = true, the table with the column family has to be created at
> peer
> cluster.
>Special cases like enabling replication at peer cluster also for cyclic
> replication has to be considered.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16379) [replication] Minor improvement to replication/copy_tables_desc.rb
[ https://issues.apache.org/jira/browse/HBASE-16379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez updated HBASE-16379: -- Attachment: 0001-HBASE-16379-replication-Minor-improvement-to-replica.patch > [replication] Minor improvement to replication/copy_tables_desc.rb > -- > > Key: HBASE-16379 > URL: https://issues.apache.org/jira/browse/HBASE-16379 > Project: HBase > Issue Type: Improvement > Components: Replication, shell >Affects Versions: 1.3.0, 1.2.2 >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez >Priority: Trivial > Attachments: > 0001-HBASE-16379-replication-Minor-improvement-to-replica.patch > > > copy_tables_desc.rb is helpful for quickly setting up a table remotely based > on an existing schema. However it does copy by default all tables. Now you > can pass a list of tables as an optional third argument and it will also > display what table descriptors where copied. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16379) [replication] Minor improvement to replication/copy_tables_desc.rb
[ https://issues.apache.org/jira/browse/HBASE-16379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez updated HBASE-16379: -- Status: Patch Available (was: Open) > [replication] Minor improvement to replication/copy_tables_desc.rb > -- > > Key: HBASE-16379 > URL: https://issues.apache.org/jira/browse/HBASE-16379 > Project: HBase > Issue Type: Improvement > Components: Replication, shell >Affects Versions: 1.2.2, 1.3.0 >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez >Priority: Trivial > Attachments: > 0001-HBASE-16379-replication-Minor-improvement-to-replica.patch > > > copy_tables_desc.rb is helpful for quickly setting up a table remotely based > on an existing schema. However it does copy by default all tables. Now you > can pass a list of tables as an optional third argument and it will also > display what table descriptors where copied. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16267) Remove commons-httpclient dependency from hbase-rest module
[
https://issues.apache.org/jira/browse/HBASE-16267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412630#comment-15412630
]
Ted Yu commented on HBASE-16267:
If there is no objection, planning to commit v14 to master branch again since
all related tests passed.
> Remove commons-httpclient dependency from hbase-rest module
> ---
>
> Key: HBASE-16267
> URL: https://issues.apache.org/jira/browse/HBASE-16267
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 16267.v10.txt, 16267.v11.txt, 16267.v12.txt,
> 16267.v13.txt, 16267.v14.txt, 16267.v2.txt, 16267.v4.txt, 16267.v6.txt,
> 16267.v8.txt, 16267.v9.txt
>
>
> hbase-rest module still has imports from org.apache.commons.httpclient .
> There is more work to be done after HBASE-15767 was integrated.
> In master branch, there seems to be transitive dependency which allows the
> code to compile:
> {code}
> [INFO] +- org.apache.hadoop:hadoop-common:jar:2.7.1:compile
> [INFO] | +- org.apache.hadoop:hadoop-annotations:jar:2.7.1:compile
> [INFO] | +- commons-cli:commons-cli:jar:1.2:compile
> [INFO] | +- org.apache.commons:commons-math3:jar:3.1.1:compile
> [INFO] | +- xmlenc:xmlenc:jar:0.52:compile
> [INFO] | +- commons-httpclient:commons-httpclient:jar:3.1:compile
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16267) Remove commons-httpclient dependency from hbase-rest module
[
https://issues.apache.org/jira/browse/HBASE-16267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412627#comment-15412627
]
Ted Yu commented on HBASE-16267:
{code}
testReversedCompleteResultWhenRegionMove(org.apache.hadoop.hbase.TestPartialResultsFromClientSide)
Time elapsed: 3.861 sec <<< ERROR!
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
attempts=36, exceptions:
Mon Aug 08 21:19:54 UTC 2016, null, java.net.SocketTimeoutException:
callTimeout=2000, callDuration=2174:
org.apache.hadoop.hbase.NotServingRegionException:
testReversedCompleteResultWhenRegionMove,,1470691190355.c6f2615b0a52b1a879e8b9298b811c0c.
is closing
at
org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:7791)
at
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2625)
at
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2620)
{code}
The above is known flaky test.
{code}
org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures Time
elapsed: 578.028 sec <<< ERROR!
org.junit.runners.model.TestTimedOutException: test timed out after 10 minutes
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1289)
{code}
Couldn't reproduce the above failure - doesn't seem to be related since only
pom.xml was changed.
{code}
Running org.apache.hadoop.hbase.http.log.TestLogLevel
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.076 sec - in
org.apache.hadoop.hbase.http.log.TestLogLevel
{code}
TestLogLevel passed.
> Remove commons-httpclient dependency from hbase-rest module
> ---
>
> Key: HBASE-16267
> URL: https://issues.apache.org/jira/browse/HBASE-16267
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 16267.v10.txt, 16267.v11.txt, 16267.v12.txt,
> 16267.v13.txt, 16267.v14.txt, 16267.v2.txt, 16267.v4.txt, 16267.v6.txt,
> 16267.v8.txt, 16267.v9.txt
>
>
> hbase-rest module still has imports from org.apache.commons.httpclient .
> There is more work to be done after HBASE-15767 was integrated.
> In master branch, there seems to be transitive dependency which allows the
> code to compile:
> {code}
> [INFO] +- org.apache.hadoop:hadoop-common:jar:2.7.1:compile
> [INFO] | +- org.apache.hadoop:hadoop-annotations:jar:2.7.1:compile
> [INFO] | +- commons-cli:commons-cli:jar:1.2:compile
> [INFO] | +- org.apache.commons:commons-math3:jar:3.1.1:compile
> [INFO] | +- xmlenc:xmlenc:jar:0.52:compile
> [INFO] | +- commons-httpclient:commons-httpclient:jar:3.1:compile
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16379) [replication] Minor improvement to replication/copy_tables_desc.rb
Esteban Gutierrez created HBASE-16379: - Summary: [replication] Minor improvement to replication/copy_tables_desc.rb Key: HBASE-16379 URL: https://issues.apache.org/jira/browse/HBASE-16379 Project: HBase Issue Type: Improvement Components: Replication, shell Affects Versions: 1.2.2, 1.3.0 Reporter: Esteban Gutierrez Assignee: Esteban Gutierrez Priority: Trivial copy_tables_desc.rb is helpful for quickly setting up a table remotely based on an existing schema. However it does copy by default all tables. Now you can pass a list of tables as an optional third argument and it will also display what table descriptors where copied. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16267) Remove commons-httpclient dependency from hbase-rest module
[
https://issues.apache.org/jira/browse/HBASE-16267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412618#comment-15412618
]
Hadoop QA commented on HBASE-16267:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 67m 6s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 1s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/0.3.0/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 3m 44s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
25s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 40s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 4s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
8s {color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s
{color} | {color:blue} Skipped patched modules with no Java source: . {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 36s
{color} | {color:red} hbase-rest in master has 1 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 13s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 52s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 47s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 47s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 9s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
32s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 57s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m 12s
{color} | {color:red} root in the patch failed. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s
{color} | {color:blue} Skipped patched modules with no Java source: . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
48s {color} | {color:green} hbase-rest generated 0 new + 0 unchanged - 1 fixed
= 0 total (was 1) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 0s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 51s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} unit {color} |
[jira] [Updated] (HBASE-16308) Contain protobuf references
[ https://issues.apache.org/jira/browse/HBASE-16308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16308: -- Attachment: HBASE-16308.master.008.patch > Contain protobuf references > --- > > Key: HBASE-16308 > URL: https://issues.apache.org/jira/browse/HBASE-16308 > Project: HBase > Issue Type: Sub-task > Components: Protobufs >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-16308.master.001.patch, > HBASE-16308.master.002.patch, HBASE-16308.master.003.patch, > HBASE-16308.master.004.patch, HBASE-16308.master.005.patch, > HBASE-16308.master.006.patch, HBASE-16308.master.006.patch, > HBASE-16308.master.007.patch, HBASE-16308.master.008.patch > > > Clean up our protobuf references so contained to just a few classes rather > than being spread about the codebase. Doing this work will make it easier > landing the parent issue and will make it more clear where the division > between shaded protobuf and unshaded protobuf lies (we need to continue with > unshaded protobuf for HDFS references by AsyncWAL and probably EndPoint > Coprocessors) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16318) fail build if license isn't in whitelist
[
https://issues.apache.org/jira/browse/HBASE-16318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412585#comment-15412585
]
Hadoop QA commented on HBASE-16318:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 17m 41s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 3m 35s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
2s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 20s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 41s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
5s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 59s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 29s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
9s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 20s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 20s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 42s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 42s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m 40s
{color} | {color:red} The patch causes 11 errors with Hadoop v2.4.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m 7s
{color} | {color:red} The patch causes 11 errors with Hadoop v2.4.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 7m 35s
{color} | {color:red} The patch causes 11 errors with Hadoop v2.5.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m 13s
{color} | {color:red} The patch causes 11 errors with Hadoop v2.5.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 13m 0s
{color} | {color:red} The patch causes 11 errors with Hadoop v2.5.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 15m 39s
{color} | {color:red} The patch causes 11 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 18m 20s
{color} | {color:red} The patch causes 11 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 20m 50s
{color} | {color:red} The patch causes 11 errors with Hadoop v2.6.3. {color} |
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m 12s
{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 18s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 34s
[jira] [Commented] (HBASE-16367) Race between master and region server initialization may lead to premature server abort
[
https://issues.apache.org/jira/browse/HBASE-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412557#comment-15412557
]
Hudson commented on HBASE-16367:
SUCCESS: Integrated in HBase-1.4 #339 (See
[https://builds.apache.org/job/HBase-1.4/339/])
HBASE-16367 Race between master and region server initialization may (tedyu:
rev 0a1a0a688363b06e3c085155992a9fb21ec2c506)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
> Race between master and region server initialization may lead to premature
> server abort
> ---
>
> Key: HBASE-16367
> URL: https://issues.apache.org/jira/browse/HBASE-16367
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 16367.addendum, 16367.v1.txt, 16367.v2.txt,
> 16367.v3.txt, 63908-master.log
>
>
> I was troubleshooting a case where hbase (1.1.2) master always dies shortly
> after start - see attached master log snippet.
> It turned out that master initialization thread was racing with
> HRegionServer#preRegistrationInitialization() (initializeZooKeeper, actually)
> since HMaster extends HRegionServer.
> Through additional logging in master:
> {code}
> this.oldLogDir = createInitialFileSystemLayout();
> HFileSystem.addLocationsOrderInterceptor(conf);
> LOG.info("creating splitLogManager");
> {code}
> I found that execution didn't reach the last log line before region server
> declared cluster Id being null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16261) MultiHFileOutputFormat Enhancement
[ https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412552#comment-15412552 ] Yi Liang commented on HBASE-16261: -- could any one help me to review the code, Thanks. :) > MultiHFileOutputFormat Enhancement > > > Key: HBASE-16261 > URL: https://issues.apache.org/jira/browse/HBASE-16261 > Project: HBase > Issue Type: Sub-task > Components: hbase >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16261-V1.patch, HBASE-16261-V2.patch, > HBASE-16261-V3.patch > > > MultiHFileOutputFormat follow HFileOutputFormat2 > (1) HFileOutputFormat2 can read one table's region split keys. and then > output multiple hfiles for one family, and each hfile map to one region. We > can add partitioner in MultiHFileOutputFormat to make it support this feature. > (2) HFileOutputFormat2 support Customized Compression algorithm for column > family and BloomFilter, also support customized DataBlockEncoding for the > output hfiles. We can also make MultiHFileOutputFormat to support these > features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16377) ServerName check is ineffective in region_mover.rb
[
https://issues.apache.org/jira/browse/HBASE-16377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412549#comment-15412549
]
Hadoop QA commented on HBASE-16377:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 1s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/0.3.0/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m 5s
{color} | {color:red} The patch generated 1 new + 323 unchanged - 1 fixed = 324
total (was 324) {color} |
| {color:green}+1{color} | {color:green} ruby-lint {color} | {color:green} 0m
2s {color} | {color:green} There were no new ruby-lint issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 20s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m 9s
{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 12s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:date2016-08-08 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12822667/16377.branch-1.v1.txt
|
| JIRA Issue | HBASE-16377 |
| Optional Tests | asflicense rubocop ruby_lint |
| uname | Linux 08fb1479acf3 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/hbase.sh |
| git revision | branch-1 / 0a1a0a6 |
| rubocop | v0.42.0 |
| rubocop |
https://builds.apache.org/job/PreCommit-HBASE-Build/3013/artifact/patchprocess/diff-patch-rubocop.txt
|
| ruby-lint | v2.3.0 |
| hbaseprotoc |
https://builds.apache.org/job/PreCommit-HBASE-Build/3013/artifact/patchprocess/patch-hbaseprotoc-root.txt
|
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3013/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> ServerName check is ineffective in region_mover.rb
> --
>
> Key: HBASE-16377
> URL: https://issues.apache.org/jira/browse/HBASE-16377
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 16377.branch-1.v1.txt, 16377.branch-1.v1.txt
>
>
> The following was observed during test of region_mover.rb :
> {code}
> 2016-08-08
> 11:17:05,341|beaver.machine|INFO|352|139926954637120|MainThread|2016-08-08
> 11:17:05,340 INFO [RubyThread-9: hbase-client/bin/thread-pool.rb:28]
> region_mover: Moving region hbase:meta,,1.1588230740 (1 of 14) from
> xYz.openstacklocal,16020,1470654716593, to
> server=xYz.openstacklocal,16020,1470654716593
> {code}
> There is check that target server should not be the same as current server:
> {code}
> if currentServer and currentServer == servername
> $LOG.info("Region " + r.getRegionNameAsString() + " (" + counter.to_s +
> " of " + regions.length.to_s + ") already on target server=" +
> servername)
> counter = counter + 1
> next
> end
> {code}
> However, the check is not effective.
> See comparison between object1 and object3:
> http://www.skorks.com/2009/09/ruby-equality-and-object-comparison/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16378) Procedure v2 - Make ProcedureException extend HBaseException
Matteo Bertozzi created HBASE-16378: --- Summary: Procedure v2 - Make ProcedureException extend HBaseException Key: HBASE-16378 URL: https://issues.apache.org/jira/browse/HBASE-16378 Project: HBase Issue Type: Sub-task Components: proc-v2 Affects Versions: 1.2.2, 1.1.5, 2.0.0, 1.3.0 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Trivial Fix For: 2.0.0, 1.3.0, 1.1.6, 1.2.3 Make ProcedureException extend HBaseException, so we can avoid stuff like HBASE-15591. and avoid try/catch ProcedureException and direct rethrows -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16370) Exclude jdk tools.jar from Bytecode Version enforcer
[
https://issues.apache.org/jira/browse/HBASE-16370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412544#comment-15412544
]
Jerry He commented on HBASE-16370:
--
Thanks [~busbey]
I was trying to cross-jdk build the shaded client (need the -Prelease) for a
customer.
Cross-jdk compilation is a tricky thing. I am rethinking if the risk-reward
worth it and overall worth it or not to make this change.
The the animal-sniffer plugin is a good thing to have. I can open the JIRA
nevertheless.
> Exclude jdk tools.jar from Bytecode Version enforcer
>
>
> Key: HBASE-16370
> URL: https://issues.apache.org/jira/browse/HBASE-16370
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.2
>Reporter: Jerry He
>Assignee: Jerry He
>Priority: Minor
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.3
>
> Attachments: HBASE-16370.patch
>
>
> Getting this message trying to do a build with -Prelease:
> {noformat}
> [INFO] Restricted to JDK 1.7 yet jdk.tools:jdk.tools:jar:1.8:system contains
> org/relaxng/datatype/DatatypeLibrary.class targeted to JDK 1.8
> [WARNING] Rule 3: org.apache.maven.plugins.enforcer.EnforceBytecodeVersion
> failed with message:
> HBase has unsupported dependencies.
> HBase requires that all dependencies be compiled with version 1.7 or earlier
> of the JDK to properly build from source. You appear to be using a newer
> dependency. You can use
> either "mvn -version" or "mvn enforcer:display-info" to verify what version
> is active.
> Non-release builds can temporarily build with a newer JDK version by
> setting the
> 'compileSource' property (eg. mvn -DcompileSource=1.8 clean package).
> Found Banned Dependency: jdk.tools:jdk.tools:jar:1.8
> Use 'mvn dependency:tree' to locate the source of the banned dependencies.
> [INFO]
>
> {noformat}
> My JDK is 1.8. But I wanted to build to target 1.7. So I didn't' have the
> -DcompileSource=1.8.
> The enforcer checks the jdk tools.jar and causes the error because the system
> JDK is 1.8.
> This is a valid build/release use case as long as we support both 1.8 and 1.7.
> We should exclude jdk tools.jar from the enforcer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16378) Procedure v2 - Make ProcedureException extend HBaseException
[ https://issues.apache.org/jira/browse/HBASE-16378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-16378: Attachment: HBASE-16378-v0.patch > Procedure v2 - Make ProcedureException extend HBaseException > > > Key: HBASE-16378 > URL: https://issues.apache.org/jira/browse/HBASE-16378 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0, 1.3.0, 1.1.5, 1.2.2 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0, 1.3.0, 1.1.6, 1.2.3 > > Attachments: HBASE-16378-v0.patch > > > Make ProcedureException extend HBaseException, so we can avoid stuff like > HBASE-15591. and avoid try/catch ProcedureException and direct rethrows -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16318) fail build if license isn't in whitelist
[ https://issues.apache.org/jira/browse/HBASE-16318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-16318: Attachment: HBASE-16318.2.patch -02 - add ALv2 name correction for Hadoop dependencies > fail build if license isn't in whitelist > > > Key: HBASE-16318 > URL: https://issues.apache.org/jira/browse/HBASE-16318 > Project: HBase > Issue Type: Sub-task > Components: build, dependencies >Reporter: Sean Busbey >Assignee: Sean Busbey > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 1.2.3, 0.98.22 > > Attachments: HBASE-16318.0.patch, HBASE-16318.1.patch, > HBASE-16318.2.patch > > > we use supplemental-models.xml to make sure we have consistent names and > descriptions for licenses. we also know what licenses we expect to see in our > build. If we see a different one > # fail the velocity template process > # if possible, include some information about why this happened -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16318) fail build if license isn't in whitelist
[ https://issues.apache.org/jira/browse/HBASE-16318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-16318: Status: Patch Available (was: In Progress) > fail build if license isn't in whitelist > > > Key: HBASE-16318 > URL: https://issues.apache.org/jira/browse/HBASE-16318 > Project: HBase > Issue Type: Sub-task > Components: build, dependencies >Reporter: Sean Busbey >Assignee: Sean Busbey > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 1.2.3, 0.98.22 > > Attachments: HBASE-16318.0.patch, HBASE-16318.1.patch, > HBASE-16318.2.patch > > > we use supplemental-models.xml to make sure we have consistent names and > descriptions for licenses. we also know what licenses we expect to see in our > build. If we see a different one > # fail the velocity template process > # if possible, include some information about why this happened -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16377) ServerName check is ineffective in region_mover.rb
[
https://issues.apache.org/jira/browse/HBASE-16377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-16377:
---
Attachment: 16377.branch-1.v1.txt
> ServerName check is ineffective in region_mover.rb
> --
>
> Key: HBASE-16377
> URL: https://issues.apache.org/jira/browse/HBASE-16377
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 16377.branch-1.v1.txt, 16377.branch-1.v1.txt
>
>
> The following was observed during test of region_mover.rb :
> {code}
> 2016-08-08
> 11:17:05,341|beaver.machine|INFO|352|139926954637120|MainThread|2016-08-08
> 11:17:05,340 INFO [RubyThread-9: hbase-client/bin/thread-pool.rb:28]
> region_mover: Moving region hbase:meta,,1.1588230740 (1 of 14) from
> xYz.openstacklocal,16020,1470654716593, to
> server=xYz.openstacklocal,16020,1470654716593
> {code}
> There is check that target server should not be the same as current server:
> {code}
> if currentServer and currentServer == servername
> $LOG.info("Region " + r.getRegionNameAsString() + " (" + counter.to_s +
> " of " + regions.length.to_s + ") already on target server=" +
> servername)
> counter = counter + 1
> next
> end
> {code}
> However, the check is not effective.
> See comparison between object1 and object3:
> http://www.skorks.com/2009/09/ruby-equality-and-object-comparison/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16377) ServerName check is ineffective in region_mover.rb
[
https://issues.apache.org/jira/browse/HBASE-16377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412518#comment-15412518
]
Ted Yu commented on HBASE-16377:
{code}
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal
org.apache.hadoop:hadoop-maven-plugins:2.5.1:protoc (compile-protoc) on project
hbase-protocol: org.apache.maven.plugin.MojoExecutionException: 'protoc
--version' did not return a version
{code}
Environment issue.
> ServerName check is ineffective in region_mover.rb
> --
>
> Key: HBASE-16377
> URL: https://issues.apache.org/jira/browse/HBASE-16377
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 16377.branch-1.v1.txt
>
>
> The following was observed during test of region_mover.rb :
> {code}
> 2016-08-08
> 11:17:05,341|beaver.machine|INFO|352|139926954637120|MainThread|2016-08-08
> 11:17:05,340 INFO [RubyThread-9: hbase-client/bin/thread-pool.rb:28]
> region_mover: Moving region hbase:meta,,1.1588230740 (1 of 14) from
> xYz.openstacklocal,16020,1470654716593, to
> server=xYz.openstacklocal,16020,1470654716593
> {code}
> There is check that target server should not be the same as current server:
> {code}
> if currentServer and currentServer == servername
> $LOG.info("Region " + r.getRegionNameAsString() + " (" + counter.to_s +
> " of " + regions.length.to_s + ") already on target server=" +
> servername)
> counter = counter + 1
> next
> end
> {code}
> However, the check is not effective.
> See comparison between object1 and object3:
> http://www.skorks.com/2009/09/ruby-equality-and-object-comparison/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16377) ServerName check is ineffective in region_mover.rb
[
https://issues.apache.org/jira/browse/HBASE-16377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412514#comment-15412514
]
Hadoop QA commented on HBASE-16377:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 1s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/0.3.0/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m 5s
{color} | {color:red} The patch generated 1 new + 323 unchanged - 1 fixed = 324
total (was 324) {color} |
| {color:green}+1{color} | {color:green} ruby-lint {color} | {color:green} 0m
2s {color} | {color:green} There were no new ruby-lint issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 54s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 1m 33s
{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 20s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:date2016-08-08 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12822659/16377.branch-1.v1.txt
|
| JIRA Issue | HBASE-16377 |
| Optional Tests | asflicense rubocop ruby_lint |
| uname | Linux 7dc35a26750b 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/hbase.sh |
| git revision | branch-1 / 0a1a0a6 |
| rubocop | v0.42.0 |
| rubocop |
https://builds.apache.org/job/PreCommit-HBASE-Build/3011/artifact/patchprocess/diff-patch-rubocop.txt
|
| ruby-lint | v2.3.0 |
| hbaseprotoc |
https://builds.apache.org/job/PreCommit-HBASE-Build/3011/artifact/patchprocess/patch-hbaseprotoc-root.txt
|
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3011/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> ServerName check is ineffective in region_mover.rb
> --
>
> Key: HBASE-16377
> URL: https://issues.apache.org/jira/browse/HBASE-16377
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 16377.branch-1.v1.txt
>
>
> The following was observed during test of region_mover.rb :
> {code}
> 2016-08-08
> 11:17:05,341|beaver.machine|INFO|352|139926954637120|MainThread|2016-08-08
> 11:17:05,340 INFO [RubyThread-9: hbase-client/bin/thread-pool.rb:28]
> region_mover: Moving region hbase:meta,,1.1588230740 (1 of 14) from
> xYz.openstacklocal,16020,1470654716593, to
> server=xYz.openstacklocal,16020,1470654716593
> {code}
> There is check that target server should not be the same as current server:
> {code}
> if currentServer and currentServer == servername
> $LOG.info("Region " + r.getRegionNameAsString() + " (" + counter.to_s +
> " of " + regions.length.to_s + ") already on target server=" +
> servername)
> counter = counter + 1
> next
> end
> {code}
> However, the check is not effective.
> See comparison between object1 and object3:
> http://www.skorks.com/2009/09/ruby-equality-and-object-comparison/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16377) ServerName check is ineffective in region_mover.rb
[
https://issues.apache.org/jira/browse/HBASE-16377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-16377:
---
Status: Patch Available (was: Open)
> ServerName check is ineffective in region_mover.rb
> --
>
> Key: HBASE-16377
> URL: https://issues.apache.org/jira/browse/HBASE-16377
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 16377.branch-1.v1.txt
>
>
> The following was observed during test of region_mover.rb :
> {code}
> 2016-08-08
> 11:17:05,341|beaver.machine|INFO|352|139926954637120|MainThread|2016-08-08
> 11:17:05,340 INFO [RubyThread-9: hbase-client/bin/thread-pool.rb:28]
> region_mover: Moving region hbase:meta,,1.1588230740 (1 of 14) from
> xYz.openstacklocal,16020,1470654716593, to
> server=xYz.openstacklocal,16020,1470654716593
> {code}
> There is check that target server should not be the same as current server:
> {code}
> if currentServer and currentServer == servername
> $LOG.info("Region " + r.getRegionNameAsString() + " (" + counter.to_s +
> " of " + regions.length.to_s + ") already on target server=" +
> servername)
> counter = counter + 1
> next
> end
> {code}
> However, the check is not effective.
> See comparison between object1 and object3:
> http://www.skorks.com/2009/09/ruby-equality-and-object-comparison/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16377) ServerName check is ineffective in region_mover.rb
[
https://issues.apache.org/jira/browse/HBASE-16377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-16377:
---
Attachment: 16377.branch-1.v1.txt
Verified that String comparison works.
> ServerName check is ineffective in region_mover.rb
> --
>
> Key: HBASE-16377
> URL: https://issues.apache.org/jira/browse/HBASE-16377
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 16377.branch-1.v1.txt
>
>
> The following was observed during test of region_mover.rb :
> {code}
> 2016-08-08
> 11:17:05,341|beaver.machine|INFO|352|139926954637120|MainThread|2016-08-08
> 11:17:05,340 INFO [RubyThread-9: hbase-client/bin/thread-pool.rb:28]
> region_mover: Moving region hbase:meta,,1.1588230740 (1 of 14) from
> xYz.openstacklocal,16020,1470654716593, to
> server=xYz.openstacklocal,16020,1470654716593
> {code}
> There is check that target server should not be the same as current server:
> {code}
> if currentServer and currentServer == servername
> $LOG.info("Region " + r.getRegionNameAsString() + " (" + counter.to_s +
> " of " + regions.length.to_s + ") already on target server=" +
> servername)
> counter = counter + 1
> next
> end
> {code}
> However, the check is not effective.
> See comparison between object1 and object3:
> http://www.skorks.com/2009/09/ruby-equality-and-object-comparison/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16377) ServerName check is ineffective in region_mover.rb
[
https://issues.apache.org/jira/browse/HBASE-16377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-16377:
---
Description:
The following was observed during test of region_mover.rb :
{code}
2016-08-08
11:17:05,341|beaver.machine|INFO|352|139926954637120|MainThread|2016-08-08
11:17:05,340 INFO [RubyThread-9: hbase-client/bin/thread-pool.rb:28]
region_mover: Moving region hbase:meta,,1.1588230740 (1 of 14) from
xYz.openstacklocal,16020,1470654716593, to
server=xYz.openstacklocal,16020,1470654716593
{code}
There is check that target server should not be the same as current server:
{code}
if currentServer and currentServer == servername
$LOG.info("Region " + r.getRegionNameAsString() + " (" + counter.to_s +
" of " + regions.length.to_s + ") already on target server=" +
servername)
counter = counter + 1
next
end
{code}
However, the check is not effective.
See comparison between object1 and object3:
http://www.skorks.com/2009/09/ruby-equality-and-object-comparison/
was:
The following was observed during test of region_mover.rb :
{code}
2016-08-08
11:17:05,341|beaver.machine|INFO|352|139926954637120|MainThread|2016-08-08
11:17:05,340 INFO [RubyThread-9:
/usr/hdp/current/hbase-client/bin/thread-pool.rb:28] region_mover: Moving
region hbase:meta,,1.1588230740 (1 of 14) from
xYz.openstacklocal,16020,1470654716593, to
server=xYz.openstacklocal,16020,1470654716593
{code}
There is check that target server should not be the same as current server:
{code}
if currentServer and currentServer == servername
$LOG.info("Region " + r.getRegionNameAsString() + " (" + counter.to_s +
" of " + regions.length.to_s + ") already on target server=" +
servername)
counter = counter + 1
next
end
{code}
However, the check is not effective.
See comparison between object1 and object3:
http://www.skorks.com/2009/09/ruby-equality-and-object-comparison/
> ServerName check is ineffective in region_mover.rb
> --
>
> Key: HBASE-16377
> URL: https://issues.apache.org/jira/browse/HBASE-16377
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.2
>Reporter: Ted Yu
>Assignee: Ted Yu
>
> The following was observed during test of region_mover.rb :
> {code}
> 2016-08-08
> 11:17:05,341|beaver.machine|INFO|352|139926954637120|MainThread|2016-08-08
> 11:17:05,340 INFO [RubyThread-9: hbase-client/bin/thread-pool.rb:28]
> region_mover: Moving region hbase:meta,,1.1588230740 (1 of 14) from
> xYz.openstacklocal,16020,1470654716593, to
> server=xYz.openstacklocal,16020,1470654716593
> {code}
> There is check that target server should not be the same as current server:
> {code}
> if currentServer and currentServer == servername
> $LOG.info("Region " + r.getRegionNameAsString() + " (" + counter.to_s +
> " of " + regions.length.to_s + ") already on target server=" +
> servername)
> counter = counter + 1
> next
> end
> {code}
> However, the check is not effective.
> See comparison between object1 and object3:
> http://www.skorks.com/2009/09/ruby-equality-and-object-comparison/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16377) ServerName check is ineffective in region_mover.rb
Ted Yu created HBASE-16377:
--
Summary: ServerName check is ineffective in region_mover.rb
Key: HBASE-16377
URL: https://issues.apache.org/jira/browse/HBASE-16377
Project: HBase
Issue Type: Bug
Affects Versions: 1.1.2
Reporter: Ted Yu
Assignee: Ted Yu
The following was observed during test of region_mover.rb :
{code}
2016-08-08
11:17:05,341|beaver.machine|INFO|352|139926954637120|MainThread|2016-08-08
11:17:05,340 INFO [RubyThread-9:
/usr/hdp/current/hbase-client/bin/thread-pool.rb:28] region_mover: Moving
region hbase:meta,,1.1588230740 (1 of 14) from
xYz.openstacklocal,16020,1470654716593, to
server=xYz.openstacklocal,16020,1470654716593
{code}
There is check that target server should not be the same as current server:
{code}
if currentServer and currentServer == servername
$LOG.info("Region " + r.getRegionNameAsString() + " (" + counter.to_s +
" of " + regions.length.to_s + ") already on target server=" +
servername)
counter = counter + 1
next
end
{code}
However, the check is not effective.
See comparison between object1 and object3:
http://www.skorks.com/2009/09/ruby-equality-and-object-comparison/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16318) fail build if license isn't in whitelist
[ https://issues.apache.org/jira/browse/HBASE-16318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-16318: Status: In Progress (was: Patch Available) fails locally as well. looks like Hadoop fixed their reference to the Apache License in 2.7.1, but all the old versions have the wrong name. Nice catch by precommit. new patch coming. > fail build if license isn't in whitelist > > > Key: HBASE-16318 > URL: https://issues.apache.org/jira/browse/HBASE-16318 > Project: HBase > Issue Type: Sub-task > Components: build, dependencies >Reporter: Sean Busbey >Assignee: Sean Busbey > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 1.2.3, 0.98.22 > > Attachments: HBASE-16318.0.patch, HBASE-16318.1.patch > > > we use supplemental-models.xml to make sure we have consistent names and > descriptions for licenses. we also know what licenses we expect to see in our > build. If we see a different one > # fail the velocity template process > # if possible, include some information about why this happened -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15554) StoreFile$Writer.appendGeneralBloomFilter generates extra KV
[
https://issues.apache.org/jira/browse/HBASE-15554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412323#comment-15412323
]
Hadoop QA commented on HBASE-15554:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 35s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
32s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
34s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 6s
{color} | {color:red} hbase-server in master has 1 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 44s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 58s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 58s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 52s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 3s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 14s
{color} | {color:red} hbase-common-jdk1.8.0_101 with JDK v1.8.0_101 generated 1
new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 17s
{color} | {color:red} hbase-common-jdk1.7.0_101 with JDK v1.7.0_101 generated 1
new + 5 unchanged - 0 fixed = 6 total (was 5) {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 40s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 97m 15s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 150m 39s {color}
| {color:black}
[jira] [Commented] (HBASE-16318) fail build if license isn't in whitelist
[ https://issues.apache.org/jira/browse/HBASE-16318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412298#comment-15412298 ] Sean Busbey commented on HBASE-16318: - lemme see if I can capture anything from the generated LICENSE files. :/ > fail build if license isn't in whitelist > > > Key: HBASE-16318 > URL: https://issues.apache.org/jira/browse/HBASE-16318 > Project: HBase > Issue Type: Sub-task > Components: build, dependencies >Reporter: Sean Busbey >Assignee: Sean Busbey > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 1.2.3, 0.98.22 > > Attachments: HBASE-16318.0.patch, HBASE-16318.1.patch > > > we use supplemental-models.xml to make sure we have consistent names and > descriptions for licenses. we also know what licenses we expect to see in our > build. If we see a different one > # fail the velocity template process > # if possible, include some information about why this happened -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16255) Backup/Restore IT
[
https://issues.apache.org/jira/browse/HBASE-16255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412242#comment-15412242
]
Dima Spivak commented on HBASE-16255:
-
Yay! Thanks for putting this together, [~vrodionov]; I'll take a look later
today. As for expectations, I was curious to see how it behaves with fault
injection. To get monkeys working, you'd just need to have your
{{IntegrationTestBackupRestore}} extend {{IntegrationTestBase}} (see:
[IntegrationTestAcidGuarantees|https://github.com/apache/hbase/blob/master/hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestAcidGuarantees.java#L44])
and then you'd get access to all the fun monkey options.
> Backup/Restore IT
> -
>
> Key: HBASE-16255
> URL: https://issues.apache.org/jira/browse/HBASE-16255
> Project: HBase
> Issue Type: Task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Labels: backup
> Attachments: HBASE-16255-v1.patch
>
>
> Integration test for backup restore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16367) Race between master and region server initialization may lead to premature server abort
[
https://issues.apache.org/jira/browse/HBASE-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412241#comment-15412241
]
Hudson commented on HBASE-16367:
FAILURE: Integrated in HBase-1.4 #338 (See
[https://builds.apache.org/job/HBase-1.4/338/])
HBASE-16367 Race between master and region server initialization may (tedyu:
rev 225383d32105ff9893c4275543f693c21e86a852)
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
> Race between master and region server initialization may lead to premature
> server abort
> ---
>
> Key: HBASE-16367
> URL: https://issues.apache.org/jira/browse/HBASE-16367
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 16367.addendum, 16367.v1.txt, 16367.v2.txt,
> 16367.v3.txt, 63908-master.log
>
>
> I was troubleshooting a case where hbase (1.1.2) master always dies shortly
> after start - see attached master log snippet.
> It turned out that master initialization thread was racing with
> HRegionServer#preRegistrationInitialization() (initializeZooKeeper, actually)
> since HMaster extends HRegionServer.
> Through additional logging in master:
> {code}
> this.oldLogDir = createInitialFileSystemLayout();
> HFileSystem.addLocationsOrderInterceptor(conf);
> LOG.info("creating splitLogManager");
> {code}
> I found that execution didn't reach the last log line before region server
> declared cluster Id being null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15554) StoreFile$Writer.appendGeneralBloomFilter generates extra KV
[ https://issues.apache.org/jira/browse/HBASE-15554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412234#comment-15412234 ] Anoop Sam John commented on HBASE-15554: Am sorry if I was not saying it clear. I dont mean still patch is having duplicate. What I mean is when I say Iterator based HashKey, I wanted it to be single structure we use with Hash rather than byte[]/BB/Cell.. But if the algo demands an offset based byte getter am fine. bq.one is that what ever be the cell format we should finally assume the back end is KV format key only. Because the offset and length that we pass to the hash algo is assuming that it is continuous Why we need pass an offset to hash() function? We need pass HashKey. Internally the impl of HashKey has to know which byte to be returned when getters are called on it. Ya if u dont have iterator model u will have get(int) which return byte. So the Hash functions has to call get() based on relative offset eg: get(0), get(1) etc. Not like cur way of offset+1, offset+2. When the impl gets these calls, it has to convert it into absolute offsets. It is not that simple in ROW_COL case. Here based on the coming offset you have it map it which area of the Cell this belongs also. That is what I was trying to say. When get(0) or get(1) is called, those comes in rkLen part. get(2) - get(+2) these belong to rk bytes.So will have to deal some sort of math. So you really dont have to assume that the Cell is of KV serialization. Just like in the past which all bytes of Cell where , continue to use those. Am I making it clear now? It would be good if we can remove any sort of KV assumption from the code path. I think it is pending only in this Bloom area. > StoreFile$Writer.appendGeneralBloomFilter generates extra KV > > > Key: HBASE-15554 > URL: https://issues.apache.org/jira/browse/HBASE-15554 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: Vladimir Rodionov >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > Attachments: HBASE-15554.patch, HBASE-15554_10.patch, > HBASE-15554_3.patch, HBASE-15554_4.patch, HBASE-15554_6.patch, > HBASE-15554_7.patch, HBASE-15554_9.patch > > > Accounts for 10% memory allocation in compaction thread when BloomFilterType > is ROWCOL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16255) Backup/Restore IT
[ https://issues.apache.org/jira/browse/HBASE-16255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412233#comment-15412233 ] Vladimir Rodionov commented on HBASE-16255: --- [~dimaspivak], What do you expect to see in this IT? I do not think we are ready to run IT with chaos monkey yet, taking into account that there is no IT for snapshots and backup are based on snapshots. So, no monkey. > Backup/Restore IT > - > > Key: HBASE-16255 > URL: https://issues.apache.org/jira/browse/HBASE-16255 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Attachments: HBASE-16255-v1.patch > > > Integration test for backup restore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16376) Document implicit side-effects on partial results when calling Scan#setBatch(int)
Josh Elser created HBASE-16376:
--
Summary: Document implicit side-effects on partial results when
calling Scan#setBatch(int)
Key: HBASE-16376
URL: https://issues.apache.org/jira/browse/HBASE-16376
Project: HBase
Issue Type: Task
Components: API, documentation
Reporter: Josh Elser
Assignee: Josh Elser
Fix For: 2.0.0, 1.3.0, 1.2.3, 0.98.22
It was brought to my attention that the javadoc on {{Scan#setBatch(int)}} does
not inform the user that calling this method has the implicit side-effect that
the user may see partial {{Result}}s.
While the side-effect isn't necessarily surprising for developers who know how
it's implemented, but for API users, this might be a very jarring implication.
We should update the documentation on {{Scan#setBatch(int)}} to inform users
that they may see partial results if they call this method (and perhaps refer
them to the size-based {{Scan#setMaxResultSize(long)}} too).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16267) Remove commons-httpclient dependency from hbase-rest module
[
https://issues.apache.org/jira/browse/HBASE-16267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-16267:
---
Status: Patch Available (was: Reopened)
> Remove commons-httpclient dependency from hbase-rest module
> ---
>
> Key: HBASE-16267
> URL: https://issues.apache.org/jira/browse/HBASE-16267
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 16267.v10.txt, 16267.v11.txt, 16267.v12.txt,
> 16267.v13.txt, 16267.v14.txt, 16267.v2.txt, 16267.v4.txt, 16267.v6.txt,
> 16267.v8.txt, 16267.v9.txt
>
>
> hbase-rest module still has imports from org.apache.commons.httpclient .
> There is more work to be done after HBASE-15767 was integrated.
> In master branch, there seems to be transitive dependency which allows the
> code to compile:
> {code}
> [INFO] +- org.apache.hadoop:hadoop-common:jar:2.7.1:compile
> [INFO] | +- org.apache.hadoop:hadoop-annotations:jar:2.7.1:compile
> [INFO] | +- commons-cli:commons-cli:jar:1.2:compile
> [INFO] | +- org.apache.commons:commons-math3:jar:3.1.1:compile
> [INFO] | +- xmlenc:xmlenc:jar:0.52:compile
> [INFO] | +- commons-httpclient:commons-httpclient:jar:3.1:compile
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16267) Remove commons-httpclient dependency from hbase-rest module
[
https://issues.apache.org/jira/browse/HBASE-16267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-16267:
---
Attachment: 16267.v14.txt
Patch v14 drops the exclusion in pom.xml
org.apache.hadoop.hbase.rest.\* and TestLogLevel pass based on v14.
> Remove commons-httpclient dependency from hbase-rest module
> ---
>
> Key: HBASE-16267
> URL: https://issues.apache.org/jira/browse/HBASE-16267
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 16267.v10.txt, 16267.v11.txt, 16267.v12.txt,
> 16267.v13.txt, 16267.v14.txt, 16267.v2.txt, 16267.v4.txt, 16267.v6.txt,
> 16267.v8.txt, 16267.v9.txt
>
>
> hbase-rest module still has imports from org.apache.commons.httpclient .
> There is more work to be done after HBASE-15767 was integrated.
> In master branch, there seems to be transitive dependency which allows the
> code to compile:
> {code}
> [INFO] +- org.apache.hadoop:hadoop-common:jar:2.7.1:compile
> [INFO] | +- org.apache.hadoop:hadoop-annotations:jar:2.7.1:compile
> [INFO] | +- commons-cli:commons-cli:jar:1.2:compile
> [INFO] | +- org.apache.commons:commons-math3:jar:3.1.1:compile
> [INFO] | +- xmlenc:xmlenc:jar:0.52:compile
> [INFO] | +- commons-httpclient:commons-httpclient:jar:3.1:compile
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16148) Hybrid Logical Clocks(placeholder for running tests)
[ https://issues.apache.org/jira/browse/HBASE-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412165#comment-15412165 ] Sai Teja Ranuva commented on HBASE-16148: - Thank you Enis. I feel the second approach may not be friendly to the client and also it will give a way for client to manipulate with server clock(not sure if it is bad thing, but sounds like it could be exploited adversely). I will try to find a way to differentiate between client set timestamps to that of replication/wal recovery as you suggested. > Hybrid Logical Clocks(placeholder for running tests) > > > Key: HBASE-16148 > URL: https://issues.apache.org/jira/browse/HBASE-16148 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: Sai Teja Ranuva >Assignee: Sai Teja Ranuva >Priority: Minor > Labels: test-patch > Attachments: HBASE-16148.master.001.patch, > HBASE-16148.master.6.patch, HBASE-16148.master.test.1.patch, > HBASE-16148.master.test.2.patch, HBASE-16148.master.test.3.patch, > HBASE-16148.master.test.4.patch, HBASE-16148.master.test.5.patch, > HLC.1.patch, HLC.2.patch, HLC.patch > > > This JIRA is just a placeholder to test Hybrid Logical Clocks code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16148) Hybrid Logical Clocks(placeholder for running tests)
[ https://issues.apache.org/jira/browse/HBASE-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Teja Ranuva updated HBASE-16148: Status: Open (was: Patch Available) > Hybrid Logical Clocks(placeholder for running tests) > > > Key: HBASE-16148 > URL: https://issues.apache.org/jira/browse/HBASE-16148 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: Sai Teja Ranuva >Assignee: Sai Teja Ranuva >Priority: Minor > Labels: test-patch > Attachments: HBASE-16148.master.001.patch, > HBASE-16148.master.6.patch, HBASE-16148.master.test.1.patch, > HBASE-16148.master.test.2.patch, HBASE-16148.master.test.3.patch, > HBASE-16148.master.test.4.patch, HBASE-16148.master.test.5.patch, > HLC.1.patch, HLC.2.patch, HLC.patch > > > This JIRA is just a placeholder to test Hybrid Logical Clocks code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16148) Hybrid Logical Clocks(placeholder for running tests)
[ https://issues.apache.org/jira/browse/HBASE-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Teja Ranuva updated HBASE-16148: Status: Patch Available (was: Open) > Hybrid Logical Clocks(placeholder for running tests) > > > Key: HBASE-16148 > URL: https://issues.apache.org/jira/browse/HBASE-16148 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: Sai Teja Ranuva >Assignee: Sai Teja Ranuva >Priority: Minor > Labels: test-patch > Attachments: HBASE-16148.master.001.patch, > HBASE-16148.master.6.patch, HBASE-16148.master.test.1.patch, > HBASE-16148.master.test.2.patch, HBASE-16148.master.test.3.patch, > HBASE-16148.master.test.4.patch, HBASE-16148.master.test.5.patch, > HLC.1.patch, HLC.2.patch, HLC.patch > > > This JIRA is just a placeholder to test Hybrid Logical Clocks code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12408) Clarify ref guide findbugs section's use of "Apache licensed version of annotations"
[
https://issues.apache.org/jira/browse/HBASE-12408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-12408:
Assignee: (was: Sean Busbey)
Labels: beginner (was: )
Priority: Trivial (was: Minor)
could still use a link to the javadocs and the clean room implementation. but
good enough for now.
unassigning myself and moving to beginner queue.
> Clarify ref guide findbugs section's use of "Apache licensed version of
> annotations"
>
>
> Key: HBASE-12408
> URL: https://issues.apache.org/jira/browse/HBASE-12408
> Project: HBase
> Issue Type: Improvement
> Components: documentation
>Reporter: Sean Busbey
>Priority: Trivial
> Labels: beginner
>
> The ref guide has a [findbugs
> section|http://hbase.apache.org/book.html#common.patch.feedback.findbugs]
> that ends with this line
> {quote}
> It is important to use the Apache-licensed version of the annotations.
> {quote}
> without any additional context, it's hard to figure out what we're talking
> about or why. The site for Findbugs itself only ever talks about its own
> license, which is LGPL.
> I think what we're getting at is that we use the fully qualified
> SupressWarnings instead of the newer SuppressFBWarnings because we rely on
> the [cleanroom ASL reimplementation of findbugs
> annotations|https://github.com/stephenc/findbugs-annotations].
> We should clarify that and provide some links.
> * [findbugs project page|http://findbugs.sourceforge.net/]
> * [cleanroom project page|https://github.com/stephenc/findbugs-annotations]
> * [javadocs|http://stephenc.github.io/findbugs-annotations/apidocs/]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16367) Race between master and region server initialization may lead to premature server abort
[
https://issues.apache.org/jira/browse/HBASE-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-16367:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Race between master and region server initialization may lead to premature
> server abort
> ---
>
> Key: HBASE-16367
> URL: https://issues.apache.org/jira/browse/HBASE-16367
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 16367.v1.txt, 16367.v2.txt, 16367.v3.txt,
> 63908-master.log
>
>
> I was troubleshooting a case where hbase (1.1.2) master always dies shortly
> after start - see attached master log snippet.
> It turned out that master initialization thread was racing with
> HRegionServer#preRegistrationInitialization() (initializeZooKeeper, actually)
> since HMaster extends HRegionServer.
> Through additional logging in master:
> {code}
> this.oldLogDir = createInitialFileSystemLayout();
> HFileSystem.addLocationsOrderInterceptor(conf);
> LOG.info("creating splitLogManager");
> {code}
> I found that execution didn't reach the last log line before region server
> declared cluster Id being null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16360) TableMapReduceUtil addHBaseDependencyJars has the wrong class name for PrefixTreeCodec
[
https://issues.apache.org/jira/browse/HBASE-16360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412144#comment-15412144
]
Jing Pu Chen commented on HBASE-16360:
--
Hi, [~mbertozzi]. I will write the test.
I think you're right. And I will test whether it works fine if we do that. I'm
not sure about this.
> TableMapReduceUtil addHBaseDependencyJars has the wrong class name for
> PrefixTreeCodec
> --
>
> Key: HBASE-16360
> URL: https://issues.apache.org/jira/browse/HBASE-16360
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Affects Versions: 2.0.0, 1.3.0, 1.0.3, 1.4.0, 1.2.2
>Reporter: Matteo Bertozzi
>Assignee: Jing Pu Chen
>Priority: Minor
> Labels: noob
> Fix For: 2.0.0, 1.3.0, 1.0.3, 1.4.0, 1.2.2
>
> Attachments: HBASE-16360-v0.patch
>
>
> HBASE-15152 included the prefix tree module as dependency to
> TableMapReduceUtil. but the hardcoded string of the class name is wrong.
> {code}
> Class.forName("org.apache.hadoop.hbase.code.prefixtree.PrefixTreeCodec");
> {code}
> should be ".codec." instead of ".code."
> {code}
> Class.forName("org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeCodec");
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16367) Race between master and region server initialization may lead to premature server abort
[
https://issues.apache.org/jira/browse/HBASE-16367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-16367:
---
Attachment: 16367.addendum
> Race between master and region server initialization may lead to premature
> server abort
> ---
>
> Key: HBASE-16367
> URL: https://issues.apache.org/jira/browse/HBASE-16367
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.2
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.4.0
>
> Attachments: 16367.addendum, 16367.v1.txt, 16367.v2.txt,
> 16367.v3.txt, 63908-master.log
>
>
> I was troubleshooting a case where hbase (1.1.2) master always dies shortly
> after start - see attached master log snippet.
> It turned out that master initialization thread was racing with
> HRegionServer#preRegistrationInitialization() (initializeZooKeeper, actually)
> since HMaster extends HRegionServer.
> Through additional logging in master:
> {code}
> this.oldLogDir = createInitialFileSystemLayout();
> HFileSystem.addLocationsOrderInterceptor(conf);
> LOG.info("creating splitLogManager");
> {code}
> I found that execution didn't reach the last log line before region server
> declared cluster Id being null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16375) Mapreduce mini cluster using HBaseTestingUtility not setting correct resourcemanager and jobhistory webapp address of MapReduceTestingShim
Loknath Priyatham Teja Singamsetty created HBASE-16375:
---
Summary: Mapreduce mini cluster using HBaseTestingUtility not
setting correct resourcemanager and jobhistory webapp address of
MapReduceTestingShim
Key: HBASE-16375
URL: https://issues.apache.org/jira/browse/HBASE-16375
Project: HBase
Issue Type: Bug
Reporter: Loknath Priyatham Teja Singamsetty
Assignee: Loknath Priyatham Teja Singamsetty
Starting mapreduce mini cluster using HBaseTestingUtility is not setting
"yarn.resourcemanager.webapp.address" and "mapreduce.jobhistory.webapp.address"
which are required for getting the submitted yarn apps using mapreduce webapp.
These properties are not being copied from jobConf of MapReduceTestingShim
resulting in default values.
{quote}
HBaseTestingUtility.java
// Allow the user to override FS URI for this map-reduce cluster to use.
mrCluster = new MiniMRCluster(servers,
FS_URI != null ? FS_URI : FileSystem.get(conf).getUri().toString(), 1,
null, null, new JobConf(this.conf));
JobConf jobConf = MapreduceTestingShim.getJobConf(mrCluster);
if (jobConf == null) {
jobConf = mrCluster.createJobConf();
}
jobConf.set("mapreduce.cluster.local.dir",
conf.get("mapreduce.cluster.local.dir")); //Hadoop MiniMR overwrites this
while it should not
LOG.info("Mini mapreduce cluster started");
// In hadoop2, YARN/MR2 starts a mini cluster with its own conf instance
and updates settings.
// Our HBase MR jobs need several of these settings in order to properly
run. So we copy the
// necessary config properties here. YARN-129 required adding a few
properties.
conf.set("mapreduce.jobtracker.address",
jobConf.get("mapreduce.jobtracker.address"));
// this for mrv2 support; mr1 ignores this
conf.set("mapreduce.framework.name", "yarn");
conf.setBoolean("yarn.is.minicluster", true);
String rmAddress = jobConf.get("yarn.resourcemanager.address");
if (rmAddress != null) {
conf.set("yarn.resourcemanager.address", rmAddress);
}
String historyAddress = jobConf.get("mapreduce.jobhistory.address");
if (historyAddress != null) {
conf.set("mapreduce.jobhistory.address", historyAddress);
}
String schedulerAddress =
jobConf.get("yarn.resourcemanager.scheduler.address");
if (schedulerAddress != null) {
conf.set("yarn.resourcemanager.scheduler.address", schedulerAddress);
}
{quote}
As a immediate fix for phoenix e2e test to succeed, need the below lines to be
added as well
{quote}
String rmWebappAddress = jobConf.get("yarn.resourcemanager.webapp.address");
if (rmWebappAddress != null) {
conf.set("yarn.resourcemanager.webapp.address", rmWebappAddress);
}
String historyWebappAddress =
jobConf.get("mapreduce.jobhistory.webapp.address");
if (historyWebappAddress != null) {
conf.set("mapreduce.jobhistory.webapp.address", historyWebappAddress);
}
{quote}
Eventually, we should also see if we can copy over all the jobConf properties
to HBaseTestingUtility conf object.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16375) Mapreduce mini cluster using HBaseTestingUtility not setting correct resourcemanager and jobhistory webapp address of MapReduceTestingShim
[
https://issues.apache.org/jira/browse/HBASE-16375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Loknath Priyatham Teja Singamsetty updated HBASE-16375:
Priority: Minor (was: Major)
> Mapreduce mini cluster using HBaseTestingUtility not setting correct
> resourcemanager and jobhistory webapp address of MapReduceTestingShim
>
>
> Key: HBASE-16375
> URL: https://issues.apache.org/jira/browse/HBASE-16375
> Project: HBase
> Issue Type: Bug
>Reporter: Loknath Priyatham Teja Singamsetty
>Assignee: Loknath Priyatham Teja Singamsetty
>Priority: Minor
>
> Starting mapreduce mini cluster using HBaseTestingUtility is not setting
> "yarn.resourcemanager.webapp.address" and
> "mapreduce.jobhistory.webapp.address" which are required for getting the
> submitted yarn apps using mapreduce webapp. These properties are not being
> copied from jobConf of MapReduceTestingShim resulting in default values.
> {quote}
> HBaseTestingUtility.java
> // Allow the user to override FS URI for this map-reduce cluster to use.
> mrCluster = new MiniMRCluster(servers,
> FS_URI != null ? FS_URI : FileSystem.get(conf).getUri().toString(), 1,
> null, null, new JobConf(this.conf));
> JobConf jobConf = MapreduceTestingShim.getJobConf(mrCluster);
> if (jobConf == null) {
> jobConf = mrCluster.createJobConf();
> }
> jobConf.set("mapreduce.cluster.local.dir",
> conf.get("mapreduce.cluster.local.dir")); //Hadoop MiniMR overwrites
> this while it should not
> LOG.info("Mini mapreduce cluster started");
> // In hadoop2, YARN/MR2 starts a mini cluster with its own conf instance
> and updates settings.
> // Our HBase MR jobs need several of these settings in order to properly
> run. So we copy the
> // necessary config properties here. YARN-129 required adding a few
> properties.
> conf.set("mapreduce.jobtracker.address",
> jobConf.get("mapreduce.jobtracker.address"));
> // this for mrv2 support; mr1 ignores this
> conf.set("mapreduce.framework.name", "yarn");
> conf.setBoolean("yarn.is.minicluster", true);
> String rmAddress = jobConf.get("yarn.resourcemanager.address");
> if (rmAddress != null) {
> conf.set("yarn.resourcemanager.address", rmAddress);
> }
> String historyAddress = jobConf.get("mapreduce.jobhistory.address");
> if (historyAddress != null) {
> conf.set("mapreduce.jobhistory.address", historyAddress);
> }
> String schedulerAddress =
> jobConf.get("yarn.resourcemanager.scheduler.address");
> if (schedulerAddress != null) {
> conf.set("yarn.resourcemanager.scheduler.address", schedulerAddress);
> }
> {quote}
> As a immediate fix for phoenix e2e test to succeed, need the below lines to
> be added as well
> {quote}
> String rmWebappAddress =
> jobConf.get("yarn.resourcemanager.webapp.address");
> if (rmWebappAddress != null) {
> conf.set("yarn.resourcemanager.webapp.address", rmWebappAddress);
> }
> String historyWebappAddress =
> jobConf.get("mapreduce.jobhistory.webapp.address");
> if (historyWebappAddress != null) {
> conf.set("mapreduce.jobhistory.webapp.address", historyWebappAddress);
> }
> {quote}
> Eventually, we should also see if we can copy over all the jobConf properties
> to HBaseTestingUtility conf object.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16318) fail build if license isn't in whitelist
[
https://issues.apache.org/jira/browse/HBASE-16318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412131#comment-15412131
]
Hadoop QA commented on HBASE-16318:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 20s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 47s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
24s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 42s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 32s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 22s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 22s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 47s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 47s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m 32s
{color} | {color:red} The patch causes 11 errors with Hadoop v2.4.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m 59s
{color} | {color:red} The patch causes 11 errors with Hadoop v2.4.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 7m 27s
{color} | {color:red} The patch causes 11 errors with Hadoop v2.5.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 9m 55s
{color} | {color:red} The patch causes 11 errors with Hadoop v2.5.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 12m 24s
{color} | {color:red} The patch causes 11 errors with Hadoop v2.5.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 14m 53s
{color} | {color:red} The patch causes 11 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 17m 23s
{color} | {color:red} The patch causes 11 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 19m 51s
{color} | {color:red} The patch causes 11 errors with Hadoop v2.6.3. {color} |
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m 10s
{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 43s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 30s
