date:20170705

[jira] [Commented] (HBASE-18319) Implement getClusterStatus/getRegionLoad/getCompactionState/getLastMajorCompactionTimestamp methods

2017-07-05 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075979#comment-16075979
 ] 

Hadoop QA commented on HBASE-18319:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 3 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
26s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  0m 
54s{color} | {color:red} hbase-client in master has 4 extant Findbugs warnings. 
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  2m 
59s{color} | {color:red} hbase-server in master has 10 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
 8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
30m 34s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m  
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
39s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}124m 21s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
34s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}178m 15s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.TestRegionLoad |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 |
| JIRA Issue | HBASE-18319 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12875847/HBASE-18319.master.002.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux 3f02b250e403 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
 |
| git revision | master / b715091 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| findbugs |

[jira] [Updated] (HBASE-18002) Investigate why bucket cache filling up in file mode in an exisiting file is slower

2017-07-05 Thread ramkrishna.s.vasudevan (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-18002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-18002:
---
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: (was: 2.0.0)
   2.0.0-alpha-2
   3.0.0
   Status: Resolved  (was: Patch Available)

Pushed to master and branch-2. Thanks for the reviews [~zjushch] and 
[~anoop.hbase].

> Investigate why bucket cache filling up in file mode in an exisiting file  is 
> slower
> 
>
> Key: HBASE-18002
> URL: https://issues.apache.org/jira/browse/HBASE-18002
> Project: HBase
>  Issue Type: Sub-task
>  Components: BucketCache
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18002_1.patch, HBASE-18002_1.patch, 
> HBASE-18002.patch
>
>
> This issue was observed when we recently did some tests with SSD based bucket 
> cache. Similar thing was also reported by @stack and [~danielpol] while doing 
> some of these bucket cache related testing.
> When we try to preload a bucket cache (in file mode) with a new file the 
> bucket cache fills up quite faster and there not much 'failedBlockAdditions'. 
> But when the same bucket cache is filled up with a preexisitng file ( that 
> had already some entries filled up) this time it has more 
> 'failedBlockAdditions' and the cache does not fill up faster. Investigate why 
> this happens. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18002) Investigate why bucket cache filling up in file mode in an exisiting file is slower

2017-07-05 Thread ramkrishna.s.vasudevan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075978#comment-16075978
 ] 

ramkrishna.s.vasudevan commented on HBASE-18002:


I removed the white space on commit.

> Investigate why bucket cache filling up in file mode in an exisiting file  is 
> slower
> 
>
> Key: HBASE-18002
> URL: https://issues.apache.org/jira/browse/HBASE-18002
> Project: HBase
>  Issue Type: Sub-task
>  Components: BucketCache
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18002_1.patch, HBASE-18002_1.patch, 
> HBASE-18002.patch
>
>
> This issue was observed when we recently did some tests with SSD based bucket 
> cache. Similar thing was also reported by @stack and [~danielpol] while doing 
> some of these bucket cache related testing.
> When we try to preload a bucket cache (in file mode) with a new file the 
> bucket cache fills up quite faster and there not much 'failedBlockAdditions'. 
> But when the same bucket cache is filled up with a preexisitng file ( that 
> had already some entries filled up) this time it has more 
> 'failedBlockAdditions' and the cache does not fill up faster. Investigate why 
> this happens. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18002) Investigate why bucket cache filling up in file mode in an exisiting file is slower

2017-07-05 Thread ramkrishna.s.vasudevan (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-18002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-18002:
---
Summary: Investigate why bucket cache filling up in file mode in an 
exisiting file  is slower  (was: Investigate why bucket cache filling up in 
file mode in an exisitng file  is slower)

> Investigate why bucket cache filling up in file mode in an exisiting file  is 
> slower
> 
>
> Key: HBASE-18002
> URL: https://issues.apache.org/jira/browse/HBASE-18002
> Project: HBase
>  Issue Type: Sub-task
>  Components: BucketCache
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> Attachments: HBASE-18002_1.patch, HBASE-18002_1.patch, 
> HBASE-18002.patch
>
>
> This issue was observed when we recently did some tests with SSD based bucket 
> cache. Similar thing was also reported by @stack and [~danielpol] while doing 
> some of these bucket cache related testing.
> When we try to preload a bucket cache (in file mode) with a new file the 
> bucket cache fills up quite faster and there not much 'failedBlockAdditions'. 
> But when the same bucket cache is filled up with a preexisitng file ( that 
> had already some entries filled up) this time it has more 
> 'failedBlockAdditions' and the cache does not fill up faster. Investigate why 
> this happens. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-14070) Hybrid Logical Clocks for HBase

2017-07-05 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-14070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075966#comment-16075966
 ] 

Hudson commented on HBASE-14070:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3321 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/3321/])
HBASE-14070 - Core HLC (stack: rev 9fe94c11690891eed6470fdb0b9bfcfc9e95a888)
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreScanner.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestCopyTable.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionSplitPolicy.java
* (add) hbase-common/src/main/java/org/apache/hadoop/hbase/Clock.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java
* (edit) 
hbase-client/src/test/java/org/apache/hadoop/hbase/TestInterfaceAudienceAnnotations.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestWALReplay.java
* (add) hbase-common/src/test/java/org/apache/hadoop/hbase/TestClock.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/querymatcher/DropDeletesCompactionScanQueryMatcher.java
* (edit) 
hbase-common/src/main/java/org/apache/hadoop/hbase/SettableTimestamp.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/querymatcher/ScanQueryMatcher.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionReplayEvents.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestIncrementTimeRange.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestCoprocessorScanPolicy.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java
* (add) 
hbase-common/src/test/java/org/apache/hadoop/hbase/TestTimestampType.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/MockRegionServerServices.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingMemStore.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* (add) hbase-common/src/main/java/org/apache/hadoop/hbase/TimestampType.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableDescriptorBuilder.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWALLockup.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDefaultMemStore.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestTableName.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCellACLWithMultipleVersions.java
* (add) 
hbase-server/src/test/java/org/apache/hadoop/hbase/TestClockWithCluster.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Region.java
* (add) hbase-common/src/main/java/org/apache/hadoop/hbase/ClockType.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableDescriptor.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
Revert "HBASE-14070 - Core HLC" Revert a push too-early (stack: rev 
c5abb6cabb312a424dc14aa77055339fe5cac5f7)
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
* (delete) hbase-common/src/main/java/org/apache/hadoop/hbase/Clock.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* (edit) 
hbase-common/src/main/java/org/apache/hadoop/hbase/SettableTimestamp.java
* (delete) 
hbase-common/src/test/java/org/apache/hadoop/hbase/TestTimestampType.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWALLockup.java
* (delete) hbase-common/src/main/java/org/apache/hadoop/hbase/ClockType.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java
* (edit)

[jira] [Commented] (HBASE-17056) Remove checked in PB generated files

2017-07-05 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075968#comment-16075968
 ] 

Hudson commented on HBASE-17056:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3321 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/3321/])
HBASE-17056 Remove checked in PB generated files Selective add of (stack: rev 
df93c13fd21a3f34aa3851893d715cbc4edb555b)
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/ProtobufArrayList.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/ReplicationProtos.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/ProtocolMessageEnum.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/TextFormatEscaper.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/MapEntryLite.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/RpcCallback.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/UnknownFieldSetLite.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/ipc/protobuf/generated/TestProcedureProtos.java
* (edit) hbase-rest/pom.xml
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/BytesValueOrBuilder.java
* (edit) hbase-endpoint/pom.xml
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/Utf8.java
* (delete) 
hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RSGroupProtos.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/BoolValue.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/DescriptorProtos.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/Duration.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/EmptyOrBuilder.java
* (delete) 
hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/LoadBalancerProtos.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/GeneratedMessageLite.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/Int32Value.java
* (delete) 
hbase-protocol/src/main/java/org/apache/hadoop/hbase/ipc/protobuf/generated/TestProtos.java
* (edit) hbase-rsgroup/pom.xml
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/BlockingService.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/MixinOrBuilder.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/ClientProtos.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/Method.java
* (delete) 
hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ErrorHandlingProtos.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/AbstractParser.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/BackupProtos.java
* (delete) 
hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/ScannerMessage.java
* (delete) 
hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/CellMessage.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/LazyFieldLite.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/SmallSortedMap.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/ValueOrBuilder.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/SingleFieldBuilderV3.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/SourceContextProto.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/GeneratedMessageV3.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/EmptyProto.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/TextFormat.java
* (delete) 
hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/com/google/protobuf/RpcController.java
* (delete)

[jira] [Commented] (HBASE-18325) Disable flakey TestMasterProcedureWalLease

2017-07-05 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075967#comment-16075967
 ] 

Hudson commented on HBASE-18325:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3321 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/3321/])
HBASE-18325 Disable flakey TestMasterProcedureWalLease (stack: rev 
172c662034dddca0592740eb94e81c8c2388a2b1)
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestMasterProcedureWalLease.java


> Disable flakey TestMasterProcedureWalLease
> --
>
> Key: HBASE-18325
> URL: https://issues.apache.org/jira/browse/HBASE-18325
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18255) Time-Delayed HBase Performance Degradation with Java 7

2017-07-05 Thread Sean Busbey (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075956#comment-16075956
 ] 

Sean Busbey commented on HBASE-18255:
-

can we get a release note on this please?

> Time-Delayed HBase Performance Degradation with Java 7
> --
>
> Key: HBASE-18255
> URL: https://issues.apache.org/jira/browse/HBASE-18255
> Project: HBase
>  Issue Type: Bug
>  Components: Performance
>Affects Versions: 1.3.1, 1.2.6, 1.1.11
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Critical
>  Labels: jdk7
> Fix For: 1.4.0, 1.3.2, 1.2.7, 1.1.12
>
> Attachments: HBASE-18255-branch-1.x.v1.patch
>
>
> The good summary of the issue and provided resolution can be found in this 
> article:
> https://community.hortonworks.com/articles/105802/time-delayed-hbase-performance-degradation-with-ja.html
> In a few words, due to internal JVM 7 bug (which has been addressed only in 
> Java 8), HotSpot code cache can become full and after that ALL JIT 
> compilations get suspended indefinitely.  The default value for code cache 
> size in JVM 7 is quite low: 48MB. It is recommended to increase this value at 
> least to 256MB (default in JVM 8).
> This BUG affects only 1.x 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18255) Time-Delayed HBase Performance Degradation with Java 7

2017-07-05 Thread Sean Busbey (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-18255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-18255:

Component/s: Performance

> Time-Delayed HBase Performance Degradation with Java 7
> --
>
> Key: HBASE-18255
> URL: https://issues.apache.org/jira/browse/HBASE-18255
> Project: HBase
>  Issue Type: Bug
>  Components: Performance
>Affects Versions: 1.3.1, 1.2.6, 1.1.11
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Critical
>  Labels: jdk7
> Fix For: 1.4.0, 1.3.2, 1.2.7, 1.1.12
>
> Attachments: HBASE-18255-branch-1.x.v1.patch
>
>
> The good summary of the issue and provided resolution can be found in this 
> article:
> https://community.hortonworks.com/articles/105802/time-delayed-hbase-performance-degradation-with-ja.html
> In a few words, due to internal JVM 7 bug (which has been addressed only in 
> Java 8), HotSpot code cache can become full and after that ALL JIT 
> compilations get suspended indefinitely.  The default value for code cache 
> size in JVM 7 is quite low: 48MB. It is recommended to increase this value at 
> least to 256MB (default in JVM 8).
> This BUG affects only 1.x 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18255) Time-Delayed HBase Performance Degradation with Java 7

2017-07-05 Thread Sean Busbey (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-18255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-18255:

Priority: Critical  (was: Major)

> Time-Delayed HBase Performance Degradation with Java 7
> --
>
> Key: HBASE-18255
> URL: https://issues.apache.org/jira/browse/HBASE-18255
> Project: HBase
>  Issue Type: Bug
>  Components: Performance
>Affects Versions: 1.3.1, 1.2.6, 1.1.11
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Critical
>  Labels: jdk7
> Fix For: 1.4.0, 1.3.2, 1.2.7, 1.1.12
>
> Attachments: HBASE-18255-branch-1.x.v1.patch
>
>
> The good summary of the issue and provided resolution can be found in this 
> article:
> https://community.hortonworks.com/articles/105802/time-delayed-hbase-performance-degradation-with-ja.html
> In a few words, due to internal JVM 7 bug (which has been addressed only in 
> Java 8), HotSpot code cache can become full and after that ALL JIT 
> compilations get suspended indefinitely.  The default value for code cache 
> size in JVM 7 is quite low: 48MB. It is recommended to increase this value at 
> least to 256MB (default in JVM 8).
> This BUG affects only 1.x 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18255) Time-Delayed HBase Performance Degradation with Java 7

2017-07-05 Thread Sean Busbey (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-18255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-18255:

Labels: jdk7  (was: )

> Time-Delayed HBase Performance Degradation with Java 7
> --
>
> Key: HBASE-18255
> URL: https://issues.apache.org/jira/browse/HBASE-18255
> Project: HBase
>  Issue Type: Bug
>  Components: Performance
>Affects Versions: 1.3.1, 1.2.6, 1.1.11
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Critical
>  Labels: jdk7
> Fix For: 1.4.0, 1.3.2, 1.2.7, 1.1.12
>
> Attachments: HBASE-18255-branch-1.x.v1.patch
>
>
> The good summary of the issue and provided resolution can be found in this 
> article:
> https://community.hortonworks.com/articles/105802/time-delayed-hbase-performance-degradation-with-ja.html
> In a few words, due to internal JVM 7 bug (which has been addressed only in 
> Java 8), HotSpot code cache can become full and after that ALL JIT 
> compilations get suspended indefinitely.  The default value for code cache 
> size in JVM 7 is quite low: 48MB. It is recommended to increase this value at 
> least to 256MB (default in JVM 8).
> This BUG affects only 1.x 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18324) [hbase-thirdparty] Tooling to prevent commits that mistakenly make use of unshaded protobuf/guava/etc.

2017-07-05 Thread Sean Busbey (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075944#comment-16075944
 ] 

Sean Busbey commented on HBASE-18324:
-

we should be able to reuse the same tooling we use to check for use of the 
hadoop annotations.

> [hbase-thirdparty] Tooling to prevent commits that mistakenly make use of 
> unshaded protobuf/guava/etc.
> --
>
> Key: HBASE-18324
> URL: https://issues.apache.org/jira/browse/HBASE-18324
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>
> Chatting w/ [~mdrob], he brought up the question of what if a dev makes use 
> of unshaded protobuf or guava? Afterall, the old jars (pb2.5 and hadoops 
> guava12.0 or whatever) are still on the CLASSPATH because upstream depends on 
> them.
> We could add a check as part of prebuild. It would complain if use of 
> com.google instead of org.apache.hadoop.hbase.shaded.com.google. But even 
> then, there are cases where com.google is legit (coprocessor endpoints) so it 
> would have to let these pass.
> TODO.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17908) Upgrade guava

2017-07-05 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075941#comment-16075941
 ] 

stack commented on HBASE-17908:
---

.019 rebase after big purge of checked-in files.

> Upgrade guava
> -
>
> Key: HBASE-17908
> URL: https://issues.apache.org/jira/browse/HBASE-17908
> Project: HBase
>  Issue Type: Sub-task
>  Components: dependencies
>Reporter: Balazs Meszaros
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-17908.master.001.patch, 
> HBASE-17908.master.002.patch, HBASE-17908.master.003.patch, 
> HBASE-17908.master.004.patch, HBASE-17908.master.005.patch, 
> HBASE-17908.master.006.patch, HBASE-17908.master.007.patch, 
> HBASE-17908.master.008.patch, HBASE-17908.master.009.patch, 
> HBASE-17908.master.010.patch, HBASE-17908.master.011.patch, 
> HBASE-17908.master.012.patch, HBASE-17908.master.013.patch, 
> HBASE-17908.master.013.patch, HBASE-17908.master.014.patch, 
> HBASE-17908.master.015.patch, HBASE-17908.master.015.patch, 
> HBASE-17908.master.016.patch, HBASE-17908.master.017.patch, 
> HBASE-17908.master.018.patch, HBASE-17908.master.019.patch
>
>
> Currently we are using guava 12.0.1, but the latest version is 21.0. 
> Upgrading guava is always a hassle because it is not always backward 
> compatible with itself.
> Currently I think there are to approaches:
> 1. Upgrade guava to the newest version (21.0) and shade it.
> 2. Upgrade guava to a version which does not break or builds (15.0).
> If we can update it, some dependencies should be removed: 
> commons-collections, commons-codec, ...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-17908) Upgrade guava

2017-07-05 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-17908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-17908:
--
Attachment: HBASE-17908.master.019.patch

> Upgrade guava
> -
>
> Key: HBASE-17908
> URL: https://issues.apache.org/jira/browse/HBASE-17908
> Project: HBase
>  Issue Type: Sub-task
>  Components: dependencies
>Reporter: Balazs Meszaros
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-17908.master.001.patch, 
> HBASE-17908.master.002.patch, HBASE-17908.master.003.patch, 
> HBASE-17908.master.004.patch, HBASE-17908.master.005.patch, 
> HBASE-17908.master.006.patch, HBASE-17908.master.007.patch, 
> HBASE-17908.master.008.patch, HBASE-17908.master.009.patch, 
> HBASE-17908.master.010.patch, HBASE-17908.master.011.patch, 
> HBASE-17908.master.012.patch, HBASE-17908.master.013.patch, 
> HBASE-17908.master.013.patch, HBASE-17908.master.014.patch, 
> HBASE-17908.master.015.patch, HBASE-17908.master.015.patch, 
> HBASE-17908.master.016.patch, HBASE-17908.master.017.patch, 
> HBASE-17908.master.018.patch, HBASE-17908.master.019.patch
>
>
> Currently we are using guava 12.0.1, but the latest version is 21.0. 
> Upgrading guava is always a hassle because it is not always backward 
> compatible with itself.
> Currently I think there are to approaches:
> 1. Upgrade guava to the newest version (21.0) and shade it.
> 2. Upgrade guava to a version which does not break or builds (15.0).
> If we can update it, some dependencies should be removed: 
> commons-collections, commons-codec, ...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-17056) Remove checked in PB generated files

2017-07-05 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-17056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-17056:
--
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Pushed to master and to branch-2.

> Remove checked in PB generated files 
> -
>
> Key: HBASE-17056
> URL: https://issues.apache.org/jira/browse/HBASE-17056
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Enis Soztutar
>Assignee: stack
> Fix For: 2.0.0
>
> Attachments: 
> 0002-HBASE-17056-Remove-checked-in-PB-generated-files.patch, 
> HBASE-17056.master.001.patch, HBASE-17056.master.002.patch, 
> HBASE-17056.master.003.patch
>
>
> Now that we have the new PB maven plugin, there is no need to have the PB 
> files checked in to the repo. The reason we did that was to ease up developer 
> env setup. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-17056) Remove checked in PB generated files

2017-07-05 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-17056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-17056:
--
Release Note: 
Purge all checked in generated protobuf files (30MB). Generate protobuf files 
inline with the build. Remove checked-in and patched protobuf. Get it from new 
hbase-thirdparty instead.

Side-effect: Our protobuf went from 3.1.0 to 3.3.1.

Build does not take noticeably longer (still about 2.5 minutes to do a mvn 
clean install -DskipTests).

IDEs will probably require a mvn build first else they'll complain about 
missing (generated) files.

  was:
Purge all checked in generated protobuf files. Generate protobuf files inline 
with the build. Remove checked-in and patched protobuf. Get it from new 
hbase-thirdparty instead.

Side-effect: Our protobuf went from 3.1.0 to 3.3.1.

Build does not take noticeably longer (still about 2.5 minutes to do a mvn 
clean install -DskipTests).

IDEs will probably require a mvn build first else they'll complain about 
missing (generated) files.


> Remove checked in PB generated files 
> -
>
> Key: HBASE-17056
> URL: https://issues.apache.org/jira/browse/HBASE-17056
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Enis Soztutar
>Assignee: stack
> Fix For: 2.0.0
>
> Attachments: 
> 0002-HBASE-17056-Remove-checked-in-PB-generated-files.patch, 
> HBASE-17056.master.001.patch, HBASE-17056.master.002.patch, 
> HBASE-17056.master.003.patch
>
>
> Now that we have the new PB maven plugin, there is no need to have the PB 
> files checked in to the repo. The reason we did that was to ease up developer 
> env setup. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18323) Remove multiple ACLs for the same user in kerberos

2017-07-05 Thread Shibin Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075887#comment-16075887
 ] 

Shibin Zhang commented on HBASE-18323:
--

[~elserj]  hi,  why  we use Ids.CREATOR_ALL_ACL ?  maybe ,it's reasonable for   
user to set the servcie user the same as  superuser.

If  we set the service user not the same as superuser ,  this service user may 
be add to znode acl 

> Remove multiple ACLs for the same user in kerberos
> --
>
> Key: HBASE-18323
> URL: https://issues.apache.org/jira/browse/HBASE-18323
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.2.0, 3.0.0
>Reporter: Shibin Zhang
>Priority: Minor
> Attachments: HBASE-18323.patch
>
>
> When deploy hbase in kerberos way ,there will be multiple acls in znode :
> 'world,'anyone
> : r
> 'sasl,'hbase
> : cdrwa
> 'sasl,'hbase
> : cdrwa
> I also see the related issue and apply the patch, like  
> https://issues.apache.org/jira/browse/HBASE-17717 
> but in my environment ,this situation still appear,
> After dig into the code , i found the reason in source code  ZKUtil.createAcl 
>  is
>  if (zkw.isClientReadable(node)) {
> LOG.error("isSecureZooKeeper user: clientReadable");
> acls.addAll(Ids.CREATOR_ALL_ACL);
> acls.addAll(Ids.READ_ACL_UNSAFE);
>   } else {
> LOG.error("isSecureZooKeeper user: clientReadable no");
> acls.addAll(Ids.CREATOR_ALL_ACL);
>   } 
>   acls.addAll(Ids.CREATOR_ALL_ACL);   
>   
>   Id AUTH_IDS = new Id("auth", "");
> ArrayList CREATOR_ALL_ACL = new ArrayList(Collections.singletonList(new 
> ACL(31, AUTH_IDS)));
> AUTH_IDS   with  "auth " will result  current connection auth user  add to 
> znode acl ,
> so it will appear multiple acls for same users.
> I think this line of code  we can remove  :  
> acls.addAll(Ids.CREATOR_ALL_ACL);   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17056) Remove checked in PB generated files

2017-07-05 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075883#comment-16075883
 ] 

Hadoop QA commented on HBASE-17056:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
 0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  7m 
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 44m 
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  3m 
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color} 
|
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  2m 
16s{color} | {color:red} hbase-protocol-shaded in master has 27 extant Findbugs 
warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  1m  
3s{color} | {color:red} hbase-client in master has 4 extant Findbugs warnings. 
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  2m 
54s{color} | {color:red} hbase-server in master has 10 extant Findbugs 
warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  0m 
39s{color} | {color:red} hbase-rest in master has 3 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  5m 
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  0m 
27s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  0m 
16s{color} | {color:red} hbase-rsgroup in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  0m 
15s{color} | {color:red} hbase-endpoint in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  0m 
36s{color} | {color:red} hbase-spark in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 
36s{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
26s{color} | {color:green} hbase-protocol-shaded generated 0 new + 0 unchanged 
- 8 fixed = 0 total (was 8) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
12s{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
20s{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
19s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
44s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
20s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
16s{color} | {color:green} hbase-endpoint in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
16s{color} | {color:green} hbase-examples in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
20s{color} | {color:green} hbase-rest in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} |

[jira] [Updated] (HBASE-15086) Fix findbugs complaint so yetus reports more green

2017-07-05 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-15086:
--
   Resolution: Fixed
Fix Version/s: 2.0.0
   Status: Resolved  (was: Patch Available)

Resolving. All subtasks done (thanks for kick [~mdrob])

> Fix findbugs complaint so yetus reports more green
> --
>
> Key: HBASE-15086
> URL: https://issues.apache.org/jira/browse/HBASE-15086
> Project: HBase
>  Issue Type: Umbrella
>  Components: build
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0
>
> Attachments: none_fix.branch-1.patch, none_fix.txt
>
>
> Umbrella issue for findbugs fixings. The red complaints in yetus report look 
> ugly. Lets fix. There ain't too many. This is umbrella issue. Can do a 
> component at a time.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18002) Investigate why bucket cache filling up in file mode in an exisitng file is slower

2017-07-05 Thread chunhui shen (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075868#comment-16075868
 ] 

chunhui shen commented on HBASE-18002:
--

Seems fine,  +1 for the patch

> Investigate why bucket cache filling up in file mode in an exisitng file  is 
> slower
> ---
>
> Key: HBASE-18002
> URL: https://issues.apache.org/jira/browse/HBASE-18002
> Project: HBase
>  Issue Type: Sub-task
>  Components: BucketCache
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> Attachments: HBASE-18002_1.patch, HBASE-18002_1.patch, 
> HBASE-18002.patch
>
>
> This issue was observed when we recently did some tests with SSD based bucket 
> cache. Similar thing was also reported by @stack and [~danielpol] while doing 
> some of these bucket cache related testing.
> When we try to preload a bucket cache (in file mode) with a new file the 
> bucket cache fills up quite faster and there not much 'failedBlockAdditions'. 
> But when the same bucket cache is filled up with a preexisitng file ( that 
> had already some entries filled up) this time it has more 
> 'failedBlockAdditions' and the cache does not fill up faster. Investigate why 
> this happens. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18295) The result contains the cells across different rows

2017-07-05 Thread Chia-Ping Tsai (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-18295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-18295:
---
Status: Patch Available  (was: Open)

>  The result contains the cells across different rows
> 
>
> Key: HBASE-18295
> URL: https://issues.apache.org/jira/browse/HBASE-18295
> Project: HBase
>  Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Blocker
> Fix For: 3.0.0, 1.4.0, 1.3.2, 2.0.0-alpha-2
>
> Attachments: HBASE-18295.branch-1.3.v0.patch, 
> HBASE-18295.branch-1.v0.patch, HBASE-18295.v0.patch, HBASE-18295.v1.patch, 
> HBASE-18295.v2.patch, HBASE-18295.v2.patch
>
>
> From the [flaky 
> dashboard|https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html]
> If we use the cell which won't be flushed into disk as the top cell to reopen 
> the scanners, the new top cell will change. If the new top cell is in 
> different row, the matcher will reset, and then matcher will accept the new 
> top cell...
> The TestStore# testFlushBeforeCompletingScan reproduces the bug.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18295) The result contains the cells across different rows

2017-07-05 Thread Chia-Ping Tsai (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-18295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-18295:
---
Attachment: HBASE-18295.v2.patch

>  The result contains the cells across different rows
> 
>
> Key: HBASE-18295
> URL: https://issues.apache.org/jira/browse/HBASE-18295
> Project: HBase
>  Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Blocker
> Fix For: 3.0.0, 1.4.0, 1.3.2, 2.0.0-alpha-2
>
> Attachments: HBASE-18295.branch-1.3.v0.patch, 
> HBASE-18295.branch-1.v0.patch, HBASE-18295.v0.patch, HBASE-18295.v1.patch, 
> HBASE-18295.v2.patch, HBASE-18295.v2.patch
>
>
> From the [flaky 
> dashboard|https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html]
> If we use the cell which won't be flushed into disk as the top cell to reopen 
> the scanners, the new top cell will change. If the new top cell is in 
> different row, the matcher will reset, and then matcher will accept the new 
> top cell...
> The TestStore# testFlushBeforeCompletingScan reproduces the bug.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18295) The result contains the cells across different rows

2017-07-05 Thread Chia-Ping Tsai (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075866#comment-16075866
 ] 

Chia-Ping Tsai commented on HBASE-18295:


The failed UTs are unrelated with v2.patch. They pass locally. retry v2.patch 
for master

>  The result contains the cells across different rows
> 
>
> Key: HBASE-18295
> URL: https://issues.apache.org/jira/browse/HBASE-18295
> Project: HBase
>  Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Blocker
> Fix For: 3.0.0, 1.4.0, 1.3.2, 2.0.0-alpha-2
>
> Attachments: HBASE-18295.branch-1.3.v0.patch, 
> HBASE-18295.branch-1.v0.patch, HBASE-18295.v0.patch, HBASE-18295.v1.patch, 
> HBASE-18295.v2.patch
>
>
> From the [flaky 
> dashboard|https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html]
> If we use the cell which won't be flushed into disk as the top cell to reopen 
> the scanners, the new top cell will change. If the new top cell is in 
> different row, the matcher will reset, and then matcher will accept the new 
> top cell...
> The TestStore# testFlushBeforeCompletingScan reproduces the bug.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18295) The result contains the cells across different rows

2017-07-05 Thread Chia-Ping Tsai (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-18295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-18295:
---
Status: Open  (was: Patch Available)

>  The result contains the cells across different rows
> 
>
> Key: HBASE-18295
> URL: https://issues.apache.org/jira/browse/HBASE-18295
> Project: HBase
>  Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Blocker
> Fix For: 3.0.0, 1.4.0, 1.3.2, 2.0.0-alpha-2
>
> Attachments: HBASE-18295.branch-1.3.v0.patch, 
> HBASE-18295.branch-1.v0.patch, HBASE-18295.v0.patch, HBASE-18295.v1.patch, 
> HBASE-18295.v2.patch
>
>
> From the [flaky 
> dashboard|https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html]
> If we use the cell which won't be flushed into disk as the top cell to reopen 
> the scanners, the new top cell will change. If the new top cell is in 
> different row, the matcher will reset, and then matcher will accept the new 
> top cell...
> The TestStore# testFlushBeforeCompletingScan reproduces the bug.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-15086) Fix findbugs complaint so yetus reports more green

2017-07-05 Thread Chia-Ping Tsai (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075860#comment-16075860
 ] 

Chia-Ping Tsai commented on HBASE-15086:


ya, I'm handling the warnings from spotbugs. [~md...@cloudera.com] Would you 
please take a look for the sub-tasks of HBASE-18266?

> Fix findbugs complaint so yetus reports more green
> --
>
> Key: HBASE-15086
> URL: https://issues.apache.org/jira/browse/HBASE-15086
> Project: HBase
>  Issue Type: Umbrella
>  Components: build
>Reporter: stack
>Assignee: stack
> Attachments: none_fix.branch-1.patch, none_fix.txt
>
>
> Umbrella issue for findbugs fixings. The red complaints in yetus report look 
> ugly. Lets fix. There ain't too many. This is umbrella issue. Can do a 
> component at a time.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18107) [AMv2] Remove DispatchMergingRegionsRequest & DispatchMergingRegions

2017-07-05 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075858#comment-16075858
 ] 

stack commented on HBASE-18107:
---

Excellent. My fault for not knowing better and pulling it back in in 
HBASE-14614. Thank you [~easyliangjob]

> [AMv2] Remove DispatchMergingRegionsRequest & DispatchMergingRegions
> 
>
> Key: HBASE-18107
> URL: https://issues.apache.org/jira/browse/HBASE-18107
> Project: HBase
>  Issue Type: Sub-task
>  Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: stack
>Assignee: Yi Liang
> Fix For: 2.0.0
>
>
> They don't align with how we have named the Split equivalents; i.e. 
> SplitRegion (so should be MergeRegion...). They probably have these awkward 
> names because the obvious slots are occupied... so this may not be fixable 
> but filing issue anyways.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18295) The result contains the cells across different rows

2017-07-05 Thread Chia-Ping Tsai (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075857#comment-16075857
 ] 

Chia-Ping Tsai commented on HBASE-18295:


{code}
if (needToReturn(outResult)) {
  return scannerContext.hasMoreValues();
}
{code}
Oh, we can't do that because the hasMoreValues() belong to NextState.

>  The result contains the cells across different rows
> 
>
> Key: HBASE-18295
> URL: https://issues.apache.org/jira/browse/HBASE-18295
> Project: HBase
>  Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Blocker
> Fix For: 3.0.0, 1.4.0, 1.3.2, 2.0.0-alpha-2
>
> Attachments: HBASE-18295.branch-1.3.v0.patch, 
> HBASE-18295.branch-1.v0.patch, HBASE-18295.v0.patch, HBASE-18295.v1.patch, 
> HBASE-18295.v2.patch
>
>
> From the [flaky 
> dashboard|https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html]
> If we use the cell which won't be flushed into disk as the top cell to reopen 
> the scanners, the new top cell will change. If the new top cell is in 
> different row, the matcher will reset, and then matcher will accept the new 
> top cell...
> The TestStore# testFlushBeforeCompletingScan reproduces the bug.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-14135) HBase Backup/Restore Phase 3: Merge backup images

2017-07-05 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-14135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075853#comment-16075853
 ] 

stack commented on HBASE-14135:
---

bq. Long running tests are well-known issue for us.

And? What is the intent? To change them? Thanks.

bq. No it is just a generic Job which can be configured and executed

You understand how someone might think 'MR' when they see 'Job'. Do you want to 
prevent any confusion?

bq. Yes, it is synchronous. All operations are client-driven.

And if the client goes away? They query to find running jobs?




> HBase Backup/Restore Phase 3: Merge backup images
> -
>
> Key: HBASE-14135
> URL: https://issues.apache.org/jira/browse/HBASE-14135
> Project: HBase
>  Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Blocker
>  Labels: backup
> Fix For: HBASE-7912
>
> Attachments: HBASE-14135-v1.patch, HBASE-14135-v3.patch
>
>
> User can merge incremental backup images into single incremental backup image.
> # Merge supports only incremental images
> # Merge supports only images for the same backup destinations
> Command:
> {code}
> hbase backup merge image1,image2,..imageK
> {code}
> Example:
> {code}
> hbase backup merge backup_143126764557,backup_143126764456 
> {code}
> When operation is complete, only the most recent backup image will be kept 
> (in above example -  backup_143126764557) as a merged backup image, all other 
> images will be deleted from both: file system and backup system tables, 
> corresponding backup manifest for the merged backup image will be updated to 
> remove dependencies from deleted images. Merged backup image will contains 
> all the data from original image and from deleted images.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18295) The result contains the cells across different rows

2017-07-05 Thread Chia-Ping Tsai (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075851#comment-16075851
 ] 

Chia-Ping Tsai commented on HBASE-18295:


bq. So even after filter, we return SEEK_NEXT_ROW or SEEK_NEXT_USING_HINT, 
there is no chance to change the top cell
I paste some codes from the v18.patch of HBASE-17125.
{code}
+matchCode = columns.checkVersions(cell, timestamp, typeByte, false);
+switch (matchCode) {
+  case SKIP:
+return MatchCode.SKIP;
+  case SEEK_NEXT_COL:
+return MatchCode.SEEK_NEXT_COL;
+  default:
+// It means it is INCLUDE, INCLUDE_AND_SEEK_NEXT_COL or 
INCLUDE_AND_SEEK_NEXT_ROW.
+assert matchCode == MatchCode.INCLUDE || matchCode == 
MatchCode.INCLUDE_AND_SEEK_NEXT_COL
+|| matchCode == MatchCode.INCLUDE_AND_SEEK_NEXT_ROW;
+break;
+}
+
+return filter == null ? matchCode : mergeFilterResponse(cell, matchCode,
+  filter.filterKeyValue(cell));
{code}
If columns#checkVersions() says SEEK_NEXT_COLUMN, we will return it directly 
w/o filter response. The UTs in this jira have columns#checkVersions() say 
SEEK_NEXT_COLUMN for triggering the bugs so i said "the filter.filterKeyValue 
isn't evaluated".

bq. how about setScannerState in the needToReturn method internal and let 
needToReturn method return a bool value? Then we can skip the null check.
Good idea. Thanks.

By the way, I apply v18.patch of HBASE-17125 and v2.patch of this jira. Then, 
the testFlushBeforeCompletingScanWithFilter fails.

>  The result contains the cells across different rows
> 
>
> Key: HBASE-18295
> URL: https://issues.apache.org/jira/browse/HBASE-18295
> Project: HBase
>  Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Blocker
> Fix For: 3.0.0, 1.4.0, 1.3.2, 2.0.0-alpha-2
>
> Attachments: HBASE-18295.branch-1.3.v0.patch, 
> HBASE-18295.branch-1.v0.patch, HBASE-18295.v0.patch, HBASE-18295.v1.patch, 
> HBASE-18295.v2.patch
>
>
> From the [flaky 
> dashboard|https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html]
> If we use the cell which won't be flushed into disk as the top cell to reopen 
> the scanners, the new top cell will change. If the new top cell is in 
> different row, the matcher will reset, and then matcher will accept the new 
> top cell...
> The TestStore# testFlushBeforeCompletingScan reproduces the bug.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-18325) Disable flakey TestMasterProcedureWalLease

2017-07-05 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-18325.
---
   Resolution: Fixed
 Assignee: stack
Fix Version/s: 2.0.0

Disable test that is failing 80% of time according to flakey test dashboard. 
Need to fix it.  HBASE-18326

> Disable flakey TestMasterProcedureWalLease
> --
>
> Key: HBASE-18325
> URL: https://issues.apache.org/jira/browse/HBASE-18325
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18326) Fix and reenable TestMasterProcedureWalLease

2017-07-05 Thread stack (JIRA)

stack created HBASE-18326:
-

 Summary: Fix and reenable TestMasterProcedureWalLease
 Key: HBASE-18326
 URL: https://issues.apache.org/jira/browse/HBASE-18326
 Project: HBase
  Issue Type: Sub-task
Reporter: stack
Priority: Blocker


Fix and reenable flakey important test.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18325) Disable flakey TestMasterProcedureWalLease

2017-07-05 Thread stack (JIRA)

stack created HBASE-18325:
-

 Summary: Disable flakey TestMasterProcedureWalLease
 Key: HBASE-18325
 URL: https://issues.apache.org/jira/browse/HBASE-18325
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18229) create new Async Split API to embrace AM v2

2017-07-05 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-18229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-18229:
--
Attachment: hbase-18229-master-v6.patch

Retry

> create new Async Split API to embrace AM v2
> ---
>
> Key: HBASE-18229
> URL: https://issues.apache.org/jira/browse/HBASE-18229
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Reporter: Yi Liang
>Assignee: Yi Liang
>Priority: Critical
> Fix For: 2.0.0-alpha-2
>
> Attachments: HBase-18229-master-v1.patch, 
> hbase-18229-master-v2.patch, HBASE-18229-master-v3.patch, 
> hbase-18229-master-v4.patch, hbase-18229-master-v5.patch, 
> hbase-18229-master-v6.patch, hbase-18229-master-v6.patch
>
>
> See HBASE-18105 for related information,  this jira will change the logic of 
> Path 1 in splitProcedure, the execution path will be:
> *HBaseAdmin.splitRegionAsync -> MasterKeepAliveConnection.splitRegion  ->  
> MasterRpcServices.splitRegion  ->  HMaster.splitRegion-> 
> AssignmentManager.submitProcedure*
> Master Will no longer ask Rs and then Rs turn around to ask master to do the 
> split operation. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18177) FanOutOneBlockAsyncDFSOutputHelper fails to compile against Hadoop 3

2017-07-05 Thread Duo Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075845#comment-16075845
 ] 

Duo Zhang commented on HBASE-18177:
---

+1.

> FanOutOneBlockAsyncDFSOutputHelper fails to compile against Hadoop 3
> 
>
> Key: HBASE-18177
> URL: https://issues.apache.org/jira/browse/HBASE-18177
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Reporter: Esteban Gutierrez
>Assignee: Mike Drob
> Fix For: 2.0.0-alpha-2
>
> Attachments: HBASE-18177.patch, HBASE-18177.v2.patch
>
>
> After HDFS-10996 ClientProtocol#create() needs to specify the erasure code 
> policy to use. In the meantime we should add a workaround to 
> FanOutOneBlockAsyncDFSOutputHelper to be able to compile against Hadoop 3 and 
> Hadoop 2.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18319) Implement getClusterStatus/getRegionLoad/getCompactionState/getLastMajorCompactionTimestamp methods

2017-07-05 Thread Guanghao Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075841#comment-16075841
 ] 

Guanghao Zhang commented on HBASE-18319:


Attach a v2 patch to fix ut.

> Implement 
> getClusterStatus/getRegionLoad/getCompactionState/getLastMajorCompactionTimestamp
>  methods
> ---
>
> Key: HBASE-18319
> URL: https://issues.apache.org/jira/browse/HBASE-18319
> Project: HBase
>  Issue Type: Sub-task
>  Components: Client
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18319.master.001.patch, 
> HBASE-18319.master.002.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18319) Implement getClusterStatus/getRegionLoad/getCompactionState/getLastMajorCompactionTimestamp methods

2017-07-05 Thread Guanghao Zhang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-18319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-18319:
---
Attachment: HBASE-18319.master.002.patch

> Implement 
> getClusterStatus/getRegionLoad/getCompactionState/getLastMajorCompactionTimestamp
>  methods
> ---
>
> Key: HBASE-18319
> URL: https://issues.apache.org/jira/browse/HBASE-18319
> Project: HBase
>  Issue Type: Sub-task
>  Components: Client
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18319.master.001.patch, 
> HBASE-18319.master.002.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18083) Make large/small file clean thread number configurable in HFileCleaner

2017-07-05 Thread Chia-Ping Tsai (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075840#comment-16075840
 ] 

Chia-Ping Tsai commented on HBASE-18083:


v2 LGTM. I had a question shown below.
{noformat}
If all configs used by HFileCleaner aren't changed, is HFileCleaner still reset 
when onConfigurationChange is called?
{noformat}
Is the cost of reset is low? 

> Make large/small file clean thread number configurable in HFileCleaner
> --
>
> Key: HBASE-18083
> URL: https://issues.apache.org/jira/browse/HBASE-18083
> Project: HBase
>  Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 3.0.0
>
> Attachments: HBASE-18083.patch, HBASE-18083.v2.patch
>
>
> Currently we have only one thread for both large and small file cleaning, but 
> when write pressure is huge we might need more cleaner threads, so we need to 
> make the thread number configurable.
> We observed more than 1.8PB data in archive directory online due to business 
> access rate change, and this proposal is one of the necessary changes 
> required.
> Default value of the configurations would still be left to 1 to keep low 
> pressure to NN for normal case.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18083) Make large/small file clean thread number configurable in HFileCleaner

2017-07-05 Thread Yu Li (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075836#comment-16075836
 ] 

Yu Li commented on HBASE-18083:
---

The failed UT cases are irrelative to change here.

[~chia7712] does the v2 patch look good to you? Thanks.

> Make large/small file clean thread number configurable in HFileCleaner
> --
>
> Key: HBASE-18083
> URL: https://issues.apache.org/jira/browse/HBASE-18083
> Project: HBase
>  Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 3.0.0
>
> Attachments: HBASE-18083.patch, HBASE-18083.v2.patch
>
>
> Currently we have only one thread for both large and small file cleaning, but 
> when write pressure is huge we might need more cleaner threads, so we need to 
> make the thread number configurable.
> We observed more than 1.8PB data in archive directory online due to business 
> access rate change, and this proposal is one of the necessary changes 
> required.
> Default value of the configurations would still be left to 1 to keep low 
> pressure to NN for normal case.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18295) The result contains the cells across different rows

2017-07-05 Thread Guanghao Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075799#comment-16075799
 ] 

Guanghao Zhang commented on HBASE-18295:


For v2 patch, how about setScannerState in the needToReturn method internal and 
let needToReturn method return a bool value? Then we can skip the null check. 
Thanks.
{code}
if (needToReturn(outResult)) {
  return scannerContext.hasMoreValues();
}
{code}

>  The result contains the cells across different rows
> 
>
> Key: HBASE-18295
> URL: https://issues.apache.org/jira/browse/HBASE-18295
> Project: HBase
>  Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Blocker
> Fix For: 3.0.0, 1.4.0, 1.3.2, 2.0.0-alpha-2
>
> Attachments: HBASE-18295.branch-1.3.v0.patch, 
> HBASE-18295.branch-1.v0.patch, HBASE-18295.v0.patch, HBASE-18295.v1.patch, 
> HBASE-18295.v2.patch
>
>
> From the [flaky 
> dashboard|https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html]
> If we use the cell which won't be flushed into disk as the top cell to reopen 
> the scanners, the new top cell will change. If the new top cell is in 
> different row, the matcher will reset, and then matcher will accept the new 
> top cell...
> The TestStore# testFlushBeforeCompletingScan reproduces the bug.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18229) create new Async Split API to embrace AM v2

2017-07-05 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075798#comment-16075798
 ] 

Hadoop QA commented on HBASE-18229:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue}  0m  
1s{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue}  0m  
1s{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 4 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 14m 
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
59s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  2m 
34s{color} | {color:red} hbase-protocol-shaded in master has 27 extant Findbugs 
warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  0m 
53s{color} | {color:red} hbase-client in master has 4 extant Findbugs warnings. 
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  2m 
36s{color} | {color:red} hbase-server in master has 10 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
6s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green}  1m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 14m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git 
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply 
{color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
32m 21s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}  
1m 38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  7m 
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
35s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  3m 
30s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}176m 23s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  8m 43s{color} 
| {color:red} hbase-shell in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  1m

[jira] [Commented] (HBASE-18295) The result contains the cells across different rows

2017-07-05 Thread Guanghao Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075783#comment-16075783
 ] 

Guanghao Zhang commented on HBASE-18295:


[~chia7712] Thanks. After HBASE-17125, if a cell be included by versions check, 
it can't be skipped by flusher, too. So even after filter, we return 
SEEK_NEXT_ROW or SEEK_NEXT_USING_HINT, there is no chance to change the top 
cell. You mean it, right?

>  The result contains the cells across different rows
> 
>
> Key: HBASE-18295
> URL: https://issues.apache.org/jira/browse/HBASE-18295
> Project: HBase
>  Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Blocker
> Fix For: 3.0.0, 1.4.0, 1.3.2, 2.0.0-alpha-2
>
> Attachments: HBASE-18295.branch-1.3.v0.patch, 
> HBASE-18295.branch-1.v0.patch, HBASE-18295.v0.patch, HBASE-18295.v1.patch, 
> HBASE-18295.v2.patch
>
>
> From the [flaky 
> dashboard|https://builds.apache.org/job/HBASE-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html]
> If we use the cell which won't be flushed into disk as the top cell to reopen 
> the scanners, the new top cell will change. If the new top cell is in 
> different row, the matcher will reset, and then matcher will accept the new 
> top cell...
> The TestStore# testFlushBeforeCompletingScan reproduces the bug.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-15631) Backport Regionserver Groups (HBASE-6721) to branch-1

2017-07-05 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-15631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075780#comment-16075780
 ] 

Andrew Purtell commented on HBASE-15631:


RB: https://reviews.apache.org/r/60672/

> Backport Regionserver Groups (HBASE-6721) to branch-1 
> --
>
> Key: HBASE-15631
> URL: https://issues.apache.org/jira/browse/HBASE-15631
> Project: HBase
>  Issue Type: New Feature
>Affects Versions: 1.4.0
>Reporter: Francis Liu
>Assignee: Francis Liu
> Attachments: HBASE-15631.02.branch-1.patch, 
> HBASE-15631_1_branch-1.patch, HBASE-15631.branch-1.1.patch, 
> HBASE-15631-branch-1.patch, HBASE-15631.branch-1.patch, HBASE-15631.patch
>
>
> Based on dev list discussion backporting region server group should not be an 
> issue as it does not: 1. destabilize the code. 2. cause backward 
> incompatibility. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-15631) Backport Regionserver Groups (HBASE-6721) to branch-1

2017-07-05 Thread Andrew Purtell (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-15631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-15631:
---
Attachment: HBASE-15631-branch-1.patch

Updated patch, incorporating:

- HBASE-15631 Backport Regionserver Groups (HBASE-6721) to branch-1 (Applied 
https://issues.apache.org/jira/secure/attachment/12799888/HBASE-15631.02.branch-1.patch)
- HBASE-17785 RSGroupBasedLoadBalancer fails to assign new table regions when 
cloning snapshot
- HBASE-15848 Fix possible null point dereference in 
RSGroupBasedLoadBalancer#getMisplacedRegions
- HBASE-15858 Some region server group shell commands don't work
- HBASE-16133 RSGroupBasedLoadBalancer.retainAssignment() might miss a region
- HBASE-16430 Fix RegionServer Group's bug when moving multiple tables
- HBASE-16456 Fix findbugs warnings in hbase-rsgroup module
- HBASE-16462 TestRSGroupsBas#testGroupBalance may hang due to uneven region 
distribution
- HBASE-17350 Fixup of regionserver group-based assignment
- HBASE-17496 RSGroup shell commands:get_server_rsgroup don't work and commands 
display an incorrect result size
- HBASE-17772 IntegrationTestRSGroup won't run
- HBASE-17758 [RSGROUP] Add shell command to move servers and tables at the 
same time
- HBASE-17806 TestRSGroups#testMoveServersAndTables is flaky in master branch
- HBASE-18235 LoadBalancer.BOGUS_SERVER_NAME should not have a bogus hostname

RSGroups and shell unit tests pass. 

> Backport Regionserver Groups (HBASE-6721) to branch-1 
> --
>
> Key: HBASE-15631
> URL: https://issues.apache.org/jira/browse/HBASE-15631
> Project: HBase
>  Issue Type: New Feature
>Affects Versions: 1.4.0
>Reporter: Francis Liu
>Assignee: Francis Liu
> Attachments: HBASE-15631.02.branch-1.patch, 
> HBASE-15631_1_branch-1.patch, HBASE-15631.branch-1.1.patch, 
> HBASE-15631-branch-1.patch, HBASE-15631.branch-1.patch, HBASE-15631.patch
>
>
> Based on dev list discussion backporting region server group should not be an 
> issue as it does not: 1. destabilize the code. 2. cause backward 
> incompatibility. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18312) Ineffective handling of FileNotFoundException in FileLink$FileLinkInputStream.tryOpen()

2017-07-05 Thread Jingcheng Du (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075772#comment-16075772
 ] 

Jingcheng Du commented on HBASE-18312:
--

+1.

> Ineffective handling of FileNotFoundException in 
> FileLink$FileLinkInputStream.tryOpen()
> ---
>
> Key: HBASE-18312
> URL: https://issues.apache.org/jira/browse/HBASE-18312
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
> Attachments: 18312.v1.txt, 18312.v2.txt, 18312.v4.txt, 18312.v5.txt, 
> 18312.v6.txt
>
>
> Found the following in region server log:
> {code}
> 2017-07-03 11:22:04,669 WARN  
> [regionserver/a.b.c.d:16020-shortCompactions-1499094046361] 
> retry.RetryInvocationHandler: Exception while   invoking 
> ClientNamenodeProtocolTranslatorPB.getBlockLocations over e.f.g.h:8020. Not 
> retrying because try once and fail.
> org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): File 
> does not exist: /hbase/data/default/X/4d61af9d1cbcc5fe2a5cbddbfc92fe7e/ 
> K/47222a9cbd294f499f49de92ecf330ee
>   at 
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:71)
>   at 
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:61)
> ...
>   at 
> org.apache.hadoop.hbase.io.FileLink$FileLinkInputStream.tryOpen(FileLink.java:291)
>   at 
> org.apache.hadoop.hbase.io.FileLink$FileLinkInputStream.(FileLink.java:122)
>   at 
> org.apache.hadoop.hbase.io.FileLink$FileLinkInputStream.(FileLink.java:113)
>   at org.apache.hadoop.hbase.io.FileLink.open(FileLink.java:404)
>   at 
> org.apache.hadoop.hbase.io.FSDataInputStreamWrapper.(FSDataInputStreamWrapper.java:98)
>   at 
> org.apache.hadoop.hbase.io.FSDataInputStreamWrapper.(FSDataInputStreamWrapper.java:83)
> {code}
> Here is related code:
> {code}
> private FSDataInputStream tryOpen() throws IOException {
>   for (Path path: fileLink.getLocations()) {
> ...
> } catch (FileNotFoundException e) {
>   // Try another file location
> }
> {code}
> The intention is to try possible locations for the linked file.
> However, RemoteException was the exception encountered. This makes the above 
> catch clause ineffective.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17826) Backup: submitting M/R job to a particular Yarn queue

2017-07-05 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075709#comment-16075709
 ] 

Hadoop QA commented on HBASE-17826:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 17m 
24s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
21s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  2m 
52s{color} | {color:red} hbase-server in master has 10 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
33m  7s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
31s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}134m  4s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
19s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}204m 17s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestReplicasClient |
|   | hadoop.hbase.master.procedure.TestMasterProcedureWalLease |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.13.1 Server=1.13.1 Image:yetus/hbase:757bf37 |
| JIRA Issue | HBASE-17826 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12875804/HBASE-17826-v1.patch |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux 05dedfdfa7b8 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / b715091 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| findbugs | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7527/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
 |
| unit | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7527/artifact/patchprocess/patch-unit-hbase-server.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7527/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7527/console |
| Powered by | Apache Yetus

[jira] [Commented] (HBASE-14135) HBase Backup/Restore Phase 3: Merge backup images

2017-07-05 Thread Vladimir Rodionov (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-14135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075705#comment-16075705
 ] 

Vladimir Rodionov commented on HBASE-14135:
---

{quote}
Why is this backup stuff in hbase-server? BackupAdmin, BackupMergeJob, etc. and 
not in hbase-backup? This is phase 3. I'd have thought the backup module would 
be well along by now (Where do I go to read on current state of backup 
feature?) hbase-server is already overly fat with test suite that takes way too 
long, etc., etc. Lets not compound.
{quote}

We have separate JIRA for modularization : HBASE-17614. This is part Phase 3. I 
can start working on that once this one is committed. Long running tests are 
well-known issue for us. and again we have a separate JIRA for that as well.

{quote}
BackupMergeJob is a MR job or something?
{quote}

No it is just a generic Job which can be configured and executed

{quote}
mergeBackups doesn't return a future? It is synchronous? Doc says nothing on 
this.
{quote}

Yes, it is synchronous.  All operations are client-driven.

Formatting will be fixed in the next patch. 


> HBase Backup/Restore Phase 3: Merge backup images
> -
>
> Key: HBASE-14135
> URL: https://issues.apache.org/jira/browse/HBASE-14135
> Project: HBase
>  Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Blocker
>  Labels: backup
> Fix For: HBASE-7912
>
> Attachments: HBASE-14135-v1.patch, HBASE-14135-v3.patch
>
>
> User can merge incremental backup images into single incremental backup image.
> # Merge supports only incremental images
> # Merge supports only images for the same backup destinations
> Command:
> {code}
> hbase backup merge image1,image2,..imageK
> {code}
> Example:
> {code}
> hbase backup merge backup_143126764557,backup_143126764456 
> {code}
> When operation is complete, only the most recent backup image will be kept 
> (in above example -  backup_143126764557) as a merged backup image, all other 
> images will be deleted from both: file system and backup system tables, 
> corresponding backup manifest for the merged backup image will be updated to 
> remove dependencies from deleted images. Merged backup image will contains 
> all the data from original image and from deleted images.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18255) Time-Delayed HBase Performance Degradation with Java 7

2017-07-05 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075680#comment-16075680
 ] 

Hudson commented on HBASE-18255:


SUCCESS: Integrated in Jenkins build HBase-1.2-JDK7 #160 (See 
[https://builds.apache.org/job/HBase-1.2-JDK7/160/])
HBASE-18255 Set JVM code cache size in provided configuration (Vladimir 
(elserj: rev 8a996e3413e93ff7b421a9a5c611ad122439d681)
* (edit) conf/hbase-env.cmd
* (edit) conf/hbase-env.sh


> Time-Delayed HBase Performance Degradation with Java 7
> --
>
> Key: HBASE-18255
> URL: https://issues.apache.org/jira/browse/HBASE-18255
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.1, 1.2.6, 1.1.11
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 1.4.0, 1.3.2, 1.2.7, 1.1.12
>
> Attachments: HBASE-18255-branch-1.x.v1.patch
>
>
> The good summary of the issue and provided resolution can be found in this 
> article:
> https://community.hortonworks.com/articles/105802/time-delayed-hbase-performance-degradation-with-ja.html
> In a few words, due to internal JVM 7 bug (which has been addressed only in 
> Java 8), HotSpot code cache can become full and after that ALL JIT 
> compilations get suspended indefinitely.  The default value for code cache 
> size in JVM 7 is quite low: 48MB. It is recommended to increase this value at 
> least to 256MB (default in JVM 8).
> This BUG affects only 1.x 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18255) Time-Delayed HBase Performance Degradation with Java 7

2017-07-05 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075656#comment-16075656
 ] 

Hudson commented on HBASE-18255:


SUCCESS: Integrated in Jenkins build HBase-1.3-JDK7 #192 (See 
[https://builds.apache.org/job/HBase-1.3-JDK7/192/])
HBASE-18255 Set JVM code cache size in provided configuration (Vladimir 
(elserj: rev 29e0e7317732e84ed02b279e4995fa012cdf041e)
* (edit) conf/hbase-env.cmd
* (edit) conf/hbase-env.sh


> Time-Delayed HBase Performance Degradation with Java 7
> --
>
> Key: HBASE-18255
> URL: https://issues.apache.org/jira/browse/HBASE-18255
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.1, 1.2.6, 1.1.11
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 1.4.0, 1.3.2, 1.2.7, 1.1.12
>
> Attachments: HBASE-18255-branch-1.x.v1.patch
>
>
> The good summary of the issue and provided resolution can be found in this 
> article:
> https://community.hortonworks.com/articles/105802/time-delayed-hbase-performance-degradation-with-ja.html
> In a few words, due to internal JVM 7 bug (which has been addressed only in 
> Java 8), HotSpot code cache can become full and after that ALL JIT 
> compilations get suspended indefinitely.  The default value for code cache 
> size in JVM 7 is quite low: 48MB. It is recommended to increase this value at 
> least to 256MB (default in JVM 8).
> This BUG affects only 1.x 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18255) Time-Delayed HBase Performance Degradation with Java 7

2017-07-05 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075645#comment-16075645
 ] 

Hudson commented on HBASE-18255:


SUCCESS: Integrated in Jenkins build HBase-1.3-JDK8 #206 (See 
[https://builds.apache.org/job/HBase-1.3-JDK8/206/])
HBASE-18255 Set JVM code cache size in provided configuration (Vladimir 
(elserj: rev 29e0e7317732e84ed02b279e4995fa012cdf041e)
* (edit) conf/hbase-env.cmd
* (edit) conf/hbase-env.sh


> Time-Delayed HBase Performance Degradation with Java 7
> --
>
> Key: HBASE-18255
> URL: https://issues.apache.org/jira/browse/HBASE-18255
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.1, 1.2.6, 1.1.11
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 1.4.0, 1.3.2, 1.2.7, 1.1.12
>
> Attachments: HBASE-18255-branch-1.x.v1.patch
>
>
> The good summary of the issue and provided resolution can be found in this 
> article:
> https://community.hortonworks.com/articles/105802/time-delayed-hbase-performance-degradation-with-ja.html
> In a few words, due to internal JVM 7 bug (which has been addressed only in 
> Java 8), HotSpot code cache can become full and after that ALL JIT 
> compilations get suspended indefinitely.  The default value for code cache 
> size in JVM 7 is quite low: 48MB. It is recommended to increase this value at 
> least to 256MB (default in JVM 8).
> This BUG affects only 1.x 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18305) Initial patch (Core HLC) for public branch

2017-07-05 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-18305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-18305:
--
Attachment: HBASE-18305.HBASE-14070.HLC.001.patch

> Initial patch (Core HLC) for public branch
> --
>
> Key: HBASE-18305
> URL: https://issues.apache.org/jira/browse/HBASE-18305
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Amit Patel
>Assignee: Amit Patel
> Attachments: HBASE-18305.HBASE-14070.HLC.001.patch, 
> HBASE-18305.master.001.patch, HBASE-18305.master.002.patch
>
>
> Used for tracking the status of the initial patch for HLC. Core HLC refers to 
> the addition of the HLC framework (i.e., Clock, Timestamp API), integration 
> of said framework with the codebase, and changes to tests. Practically all of 
> this work has been done by our [~saitejar]. 
> This patch in particular intentionally does not include:
> * undoing using the master's timestamp for meta updates,
> * enabling HLC for meta
> * updating the clock for events like recovery, replication, region 
> open/close. 
> These changes will be introduced soon in smaller followup commits.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18255) Time-Delayed HBase Performance Degradation with Java 7

2017-07-05 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075642#comment-16075642
 ] 

Hudson commented on HBASE-18255:


SUCCESS: Integrated in Jenkins build HBase-1.2-JDK8 #156 (See 
[https://builds.apache.org/job/HBase-1.2-JDK8/156/])
HBASE-18255 Set JVM code cache size in provided configuration (Vladimir 
(elserj: rev 8a996e3413e93ff7b421a9a5c611ad122439d681)
* (edit) conf/hbase-env.cmd
* (edit) conf/hbase-env.sh


> Time-Delayed HBase Performance Degradation with Java 7
> --
>
> Key: HBASE-18255
> URL: https://issues.apache.org/jira/browse/HBASE-18255
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.1, 1.2.6, 1.1.11
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 1.4.0, 1.3.2, 1.2.7, 1.1.12
>
> Attachments: HBASE-18255-branch-1.x.v1.patch
>
>
> The good summary of the issue and provided resolution can be found in this 
> article:
> https://community.hortonworks.com/articles/105802/time-delayed-hbase-performance-degradation-with-ja.html
> In a few words, due to internal JVM 7 bug (which has been addressed only in 
> Java 8), HotSpot code cache can become full and after that ALL JIT 
> compilations get suspended indefinitely.  The default value for code cache 
> size in JVM 7 is quite low: 48MB. It is recommended to increase this value at 
> least to 256MB (default in JVM 8).
> This BUG affects only 1.x 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18305) Initial patch (Core HLC) for public branch

2017-07-05 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-18305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-18305:
--
Attachment: HBASE-18305.master.002.patch

> Initial patch (Core HLC) for public branch
> --
>
> Key: HBASE-18305
> URL: https://issues.apache.org/jira/browse/HBASE-18305
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Amit Patel
>Assignee: Amit Patel
> Attachments: HBASE-18305.master.001.patch, 
> HBASE-18305.master.002.patch
>
>
> Used for tracking the status of the initial patch for HLC. Core HLC refers to 
> the addition of the HLC framework (i.e., Clock, Timestamp API), integration 
> of said framework with the codebase, and changes to tests. Practically all of 
> this work has been done by our [~saitejar]. 
> This patch in particular intentionally does not include:
> * undoing using the master's timestamp for meta updates,
> * enabling HLC for meta
> * updating the clock for events like recovery, replication, region 
> open/close. 
> These changes will be introduced soon in smaller followup commits.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17705) Procedure execution must fail fast if procedure is not registered

2017-07-05 Thread Vladimir Rodionov (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075633#comment-16075633
 ] 

Vladimir Rodionov commented on HBASE-17705:
---

{quote}
IOE is our base exception so catching it is a little indiscriminating. Can the 
test have more precision? Catch DoNotRetryIOE at least or some such?
{quote}

v3 catches DNRIOE.

> Procedure execution must fail fast if procedure is not registered
> -
>
> Key: HBASE-17705
> URL: https://issues.apache.org/jira/browse/HBASE-17705
> Project: HBase
>  Issue Type: Bug
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: HBASE-17705.v2.patch, HBASE-17705.v3.patch
>
>
> The issue was discovered during backup testing and described here:
> https://issues.apache.org/jira/browse/HBASE-14123?focusedCommentId=15886712=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15886712
> If procedure is not registered, client will be retrying until max attempts 
> reached. The Master should throw DoNotRetryIOException and client RPC should 
> handle this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-17705) Procedure execution must fail fast if procedure is not registered

2017-07-05 Thread Vladimir Rodionov (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-17705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-17705:
--
Attachment: HBASE-17705.v3.patch

> Procedure execution must fail fast if procedure is not registered
> -
>
> Key: HBASE-17705
> URL: https://issues.apache.org/jira/browse/HBASE-17705
> Project: HBase
>  Issue Type: Bug
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: HBASE-17705.v2.patch, HBASE-17705.v3.patch
>
>
> The issue was discovered during backup testing and described here:
> https://issues.apache.org/jira/browse/HBASE-14123?focusedCommentId=15886712=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15886712
> If procedure is not registered, client will be retrying until max attempts 
> reached. The Master should throw DoNotRetryIOException and client RPC should 
> handle this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18107) [AMv2] Remove DispatchMergingRegionsRequest & DispatchMergingRegions

2017-07-05 Thread Yi Liang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yi Liang updated HBASE-18107:
-
Summary: [AMv2] Remove DispatchMergingRegionsRequest & 
DispatchMergingRegions  (was: [AMv2] Rename DispatchMergingRegionsRequest & 
DispatchMergingRegions)

> [AMv2] Remove DispatchMergingRegionsRequest & DispatchMergingRegions
> 
>
> Key: HBASE-18107
> URL: https://issues.apache.org/jira/browse/HBASE-18107
> Project: HBase
>  Issue Type: Sub-task
>  Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: stack
> Fix For: 2.0.0
>
>
> They don't align with how we have named the Split equivalents; i.e. 
> SplitRegion (so should be MergeRegion...). They probably have these awkward 
> names because the obvious slots are occupied... so this may not be fixable 
> but filing issue anyways.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (HBASE-18107) [AMv2] Rename DispatchMergingRegionsRequest & DispatchMergingRegions

2017-07-05 Thread Yi Liang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075546#comment-16075546
 ] 

Yi Liang edited comment on HBASE-18107 at 7/5/17 11:35 PM:
---

Hi [~stack], below is my finds about this DispatchMergingRegions
here is the 2 execute paths of merge in two branches
{quote}
In branch-1.2:
HBaseAdmin.mergeRegions 
-> MasterKeepAliveConnection.dispatchMergingRegions
-> MasterRpcServices.dispatchMergingRegions
-> HMaster.dispatchMergingRegions
-> DispatchMergingRegionHandler
In branch master:
HBaseAdmin.mergeRegionsAsync 
-> MasterKeepAliveConnection.mergeTableRegions
-> MasterRpcServices.mergeTableRegions
-> HMaster.mergeRegions 
-> MasterProcedureUtil.submitProcedure 
{quote}

It seems that [~syuanjiang]  have replaced  dispatchMergingRegions with 
MergeTableRegionsProcedure, followed by the history of merge procedure:

In HBASE-14552,The DispatchMergingRegionHandler has been replaced with 
DispatchMergingRegionsProcedure

In HBASE-16119 DispatchMergingRegionsProcedure has been replaced with 
MergeTableRegionsProcedure

In HBASE-17470 DispatchMergingRegionsProcedure has been removed

In HBASE-14614, it seems that DispatchMergingRegionsProcedure added back again


So from my point of view, we need to remove DispatchMergingRegionsProcedure
i.e we need to remove the path 2 of Merge Regions as I mentioned in HBASE-18105





was (Author: easyliangjob):
Hi [~stack], below is my finds about this DispatchMergingRegions
here is the 2 execute paths of merge in two branches
{quote}
In branch-1.2:
HBaseAdmin.mergeRegions 
-> MasterKeepAliveConnection.dispatchMergingRegions
-> MasterRpcServices.dispatchMergingRegions
-> HMaster.dispatchMergingRegions
-> DispatchMergingRegionHandler
In branch master:
HBaseAdmin.mergeRegionsAsync 
-> MasterKeepAliveConnection.mergeTableRegions
-> MasterRpcServices.mergeTableRegions
-> HMaster.mergeRegions 
-> MasterProcedureUtil.submitProcedure 
{quote}

It seems that [~syuanjiang]  have replaced  dispatchMergingRegions with 
MergeTableRegionsProcedure, followed by the history of merge procedure:

In HBASE-14552,The DispatchMergingRegionHandler has been replaced with 
DispatchMergingRegionsProcedure

In HBASE-16119 DispatchMergingRegionsProcedure has been replaced with 
MergeTableRegionsProcedure

In HBASE-17470 DispatchMergingRegionsProcedure has been removed

In HBase-14614, it seems that DispatchMergingRegionsProcedure added back again


So from my point of view, we need to remove DispatchMergingRegionsProcedure
i.e we need to remove the path 2 of Merge Regions as I mentioned in HBASE-18105




> [AMv2] Rename DispatchMergingRegionsRequest & DispatchMergingRegions
> 
>
> Key: HBASE-18107
> URL: https://issues.apache.org/jira/browse/HBASE-18107
> Project: HBase
>  Issue Type: Sub-task
>  Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: stack
> Fix For: 2.0.0
>
>
> They don't align with how we have named the Split equivalents; i.e. 
> SplitRegion (so should be MergeRegion...). They probably have these awkward 
> names because the obvious slots are occupied... so this may not be fixable 
> but filing issue anyways.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Reopened] (HBASE-18107) [AMv2] Remove DispatchMergingRegionsRequest & DispatchMergingRegions

2017-07-05 Thread Yi Liang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yi Liang reopened HBASE-18107:
--
  Assignee: Yi Liang

> [AMv2] Remove DispatchMergingRegionsRequest & DispatchMergingRegions
> 
>
> Key: HBASE-18107
> URL: https://issues.apache.org/jira/browse/HBASE-18107
> Project: HBase
>  Issue Type: Sub-task
>  Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: stack
>Assignee: Yi Liang
> Fix For: 2.0.0
>
>
> They don't align with how we have named the Split equivalents; i.e. 
> SplitRegion (so should be MergeRegion...). They probably have these awkward 
> names because the obvious slots are occupied... so this may not be fixable 
> but filing issue anyways.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18255) Time-Delayed HBase Performance Degradation with Java 7

2017-07-05 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075628#comment-16075628
 ] 

Hudson commented on HBASE-18255:


SUCCESS: Integrated in Jenkins build HBase-1.1-JDK7 #1889 (See 
[https://builds.apache.org/job/HBase-1.1-JDK7/1889/])
HBASE-18255 Set JVM code cache size in provided configuration (Vladimir 
(elserj: rev 975eaabad0f90035bfab17be6606ca56359a97bf)
* (edit) conf/hbase-env.sh
* (edit) conf/hbase-env.cmd


> Time-Delayed HBase Performance Degradation with Java 7
> --
>
> Key: HBASE-18255
> URL: https://issues.apache.org/jira/browse/HBASE-18255
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.1, 1.2.6, 1.1.11
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 1.4.0, 1.3.2, 1.2.7, 1.1.12
>
> Attachments: HBASE-18255-branch-1.x.v1.patch
>
>
> The good summary of the issue and provided resolution can be found in this 
> article:
> https://community.hortonworks.com/articles/105802/time-delayed-hbase-performance-degradation-with-ja.html
> In a few words, due to internal JVM 7 bug (which has been addressed only in 
> Java 8), HotSpot code cache can become full and after that ALL JIT 
> compilations get suspended indefinitely.  The default value for code cache 
> size in JVM 7 is quite low: 48MB. It is recommended to increase this value at 
> least to 256MB (default in JVM 8).
> This BUG affects only 1.x 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18107) [AMv2] Rename DispatchMergingRegionsRequest & DispatchMergingRegions

2017-07-05 Thread Yi Liang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075627#comment-16075627
 ] 

Yi Liang commented on HBASE-18107:
--

Ok, I will remove it in this patch

> [AMv2] Rename DispatchMergingRegionsRequest & DispatchMergingRegions
> 
>
> Key: HBASE-18107
> URL: https://issues.apache.org/jira/browse/HBASE-18107
> Project: HBase
>  Issue Type: Sub-task
>  Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: stack
> Fix For: 2.0.0
>
>
> They don't align with how we have named the Split equivalents; i.e. 
> SplitRegion (so should be MergeRegion...). They probably have these awkward 
> names because the obvious slots are occupied... so this may not be fixable 
> but filing issue anyways.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (HBASE-18107) [AMv2] Rename DispatchMergingRegionsRequest & DispatchMergingRegions

2017-07-05 Thread Yi Liang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075627#comment-16075627
 ] 

Yi Liang edited comment on HBASE-18107 at 7/5/17 11:34 PM:
---

Ok, I will remove it in this jira


was (Author: easyliangjob):
Ok, I will remove it in this patch

> [AMv2] Rename DispatchMergingRegionsRequest & DispatchMergingRegions
> 
>
> Key: HBASE-18107
> URL: https://issues.apache.org/jira/browse/HBASE-18107
> Project: HBase
>  Issue Type: Sub-task
>  Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: stack
> Fix For: 2.0.0
>
>
> They don't align with how we have named the Split equivalents; i.e. 
> SplitRegion (so should be MergeRegion...). They probably have these awkward 
> names because the obvious slots are occupied... so this may not be fixable 
> but filing issue anyways.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18107) [AMv2] Rename DispatchMergingRegionsRequest & DispatchMergingRegions

2017-07-05 Thread Stephen Yuan Jiang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075619#comment-16075619
 ] 

Stephen Yuan Jiang commented on HBASE-18107:


Yeah, we don't need DispatchMergingRegionsProcedure in 2.0.0.  Sorry that I did 
not find this issue in HBASE-14614 when it sneak back this old procedure. 

> [AMv2] Rename DispatchMergingRegionsRequest & DispatchMergingRegions
> 
>
> Key: HBASE-18107
> URL: https://issues.apache.org/jira/browse/HBASE-18107
> Project: HBase
>  Issue Type: Sub-task
>  Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: stack
> Fix For: 2.0.0
>
>
> They don't align with how we have named the Split equivalents; i.e. 
> SplitRegion (so should be MergeRegion...). They probably have these awkward 
> names because the obvious slots are occupied... so this may not be fixable 
> but filing issue anyways.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18255) Time-Delayed HBase Performance Degradation with Java 7

2017-07-05 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075620#comment-16075620
 ] 

Hudson commented on HBASE-18255:


FAILURE: Integrated in Jenkins build HBase-1.4 #801 (See 
[https://builds.apache.org/job/HBase-1.4/801/])
HBASE-18255 Set JVM code cache size in provided configuration (Vladimir 
(elserj: rev 3a0a3fd5fe8428fe2aca953f2f83bfaddde37452)
* (edit) conf/hbase-env.cmd
* (edit) conf/hbase-env.sh


> Time-Delayed HBase Performance Degradation with Java 7
> --
>
> Key: HBASE-18255
> URL: https://issues.apache.org/jira/browse/HBASE-18255
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.1, 1.2.6, 1.1.11
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 1.4.0, 1.3.2, 1.2.7, 1.1.12
>
> Attachments: HBASE-18255-branch-1.x.v1.patch
>
>
> The good summary of the issue and provided resolution can be found in this 
> article:
> https://community.hortonworks.com/articles/105802/time-delayed-hbase-performance-degradation-with-ja.html
> In a few words, due to internal JVM 7 bug (which has been addressed only in 
> Java 8), HotSpot code cache can become full and after that ALL JIT 
> compilations get suspended indefinitely.  The default value for code cache 
> size in JVM 7 is quite low: 48MB. It is recommended to increase this value at 
> least to 256MB (default in JVM 8).
> This BUG affects only 1.x 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-12349) Add Maven build support module for a custom version of error-prone

2017-07-05 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-12349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075606#comment-16075606
 ] 

Andrew Purtell commented on HBASE-12349:


Oh that's great, because packaging up HBase build specific rules into a custom 
version was basically a nonstarter

> Add Maven build support module for a custom version of error-prone
> --
>
> Key: HBASE-12349
> URL: https://issues.apache.org/jira/browse/HBASE-12349
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Andrew Purtell
> Fix For: 2.0.0
>
>
> Add a new Maven build support module that builds and publishes a custom 
> error-prone artifact for use by the rest of the build.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18324) [hbase-thirdparty] Tooling to prevent commits that mistakenly make use of unshaded protobuf/guava/etc.

2017-07-05 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075604#comment-16075604
 ] 

Andrew Purtell commented on HBASE-18324:


Right, it would have to be a custom rule

> [hbase-thirdparty] Tooling to prevent commits that mistakenly make use of 
> unshaded protobuf/guava/etc.
> --
>
> Key: HBASE-18324
> URL: https://issues.apache.org/jira/browse/HBASE-18324
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>
> Chatting w/ [~mdrob], he brought up the question of what if a dev makes use 
> of unshaded protobuf or guava? Afterall, the old jars (pb2.5 and hadoops 
> guava12.0 or whatever) are still on the CLASSPATH because upstream depends on 
> them.
> We could add a check as part of prebuild. It would complain if use of 
> com.google instead of org.apache.hadoop.hbase.shaded.com.google. But even 
> then, there are cases where com.google is legit (coprocessor endpoints) so it 
> would have to let these pass.
> TODO.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-14379) Replication V2

2017-07-05 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075603#comment-16075603
 ] 

Andrew Purtell commented on HBASE-14379:


People are picking up pieces. Seems right not to schedule the umbrella.

> Replication V2
> --
>
> Key: HBASE-14379
> URL: https://issues.apache.org/jira/browse/HBASE-14379
> Project: HBase
>  Issue Type: Umbrella
>  Components: Replication
>Reporter: Andrew Purtell
>
> Replication V2 is a tear-down of exiting replication code to just the 
> interfaces introduced in HBASE-11367, then a rebuild around the following 
> principles, goals, and suggested features:
> - No state in ZooKeeper. Introduce a new system table for tracking peers, 
> queues, and log positions. (Some discussion on HBASE-10295, probably will be 
> replaced with a set of more focused issues.)
> - Allow replication v1 and v2 to coexist. Note all of the undesirable 
> features of v1 will remain as long as v1 is active, 'fixing' v1 is out of 
> scope. Supporting communication between v1 and v2 endpoints would also be out 
> of scope.
> - Simplified internal programming model based on iterators
> - Streaming data transfer
> - Administrative actions mediated by the master with support for security 
> hooks (like HBASE-11392)
> - Replication state persisted and communicated with protobuf (like 
> HBASE-11393 but everywhere)
> - Detailed metrics
> - Support for at least simple status checks and admin actions via UI and shell
> - Hbck support for fixing corrupt or stuck queues (like HBASE-14014)
> - Support for bulk load, perhaps through augmenting bulk load to build WALs 
> as well as HFiles (see HBASE-13153)
> - Optional consideration for replicating schema as well as data (like 
> HBASE-12947). May fall out of scope.
> - Optional separation of replication function from the regionservers (see 
> HBASE-8772)
> - Optional alternate scheduling of edits besides FIFO-by-region (see 
> HBASE-1734 and HBASE-14014)
> There are a number of existing JIRAs that will eventually be closed as 
> duplicate, wont fix, or reparented here.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-14379) Replication V2

2017-07-05 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075599#comment-16075599
 ] 

stack commented on HBASE-14379:
---

Unscheduling feature not being worked on (that I know of).

> Replication V2
> --
>
> Key: HBASE-14379
> URL: https://issues.apache.org/jira/browse/HBASE-14379
> Project: HBase
>  Issue Type: Umbrella
>  Components: Replication
>Reporter: Andrew Purtell
>
> Replication V2 is a tear-down of exiting replication code to just the 
> interfaces introduced in HBASE-11367, then a rebuild around the following 
> principles, goals, and suggested features:
> - No state in ZooKeeper. Introduce a new system table for tracking peers, 
> queues, and log positions. (Some discussion on HBASE-10295, probably will be 
> replaced with a set of more focused issues.)
> - Allow replication v1 and v2 to coexist. Note all of the undesirable 
> features of v1 will remain as long as v1 is active, 'fixing' v1 is out of 
> scope. Supporting communication between v1 and v2 endpoints would also be out 
> of scope.
> - Simplified internal programming model based on iterators
> - Streaming data transfer
> - Administrative actions mediated by the master with support for security 
> hooks (like HBASE-11392)
> - Replication state persisted and communicated with protobuf (like 
> HBASE-11393 but everywhere)
> - Detailed metrics
> - Support for at least simple status checks and admin actions via UI and shell
> - Hbck support for fixing corrupt or stuck queues (like HBASE-14014)
> - Support for bulk load, perhaps through augmenting bulk load to build WALs 
> as well as HFiles (see HBASE-13153)
> - Optional consideration for replicating schema as well as data (like 
> HBASE-12947). May fall out of scope.
> - Optional separation of replication function from the regionservers (see 
> HBASE-8772)
> - Optional alternate scheduling of edits besides FIFO-by-region (see 
> HBASE-1734 and HBASE-14014)
> There are a number of existing JIRAs that will eventually be closed as 
> duplicate, wont fix, or reparented here.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-14379) Replication V2

2017-07-05 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-14379:
--
Fix Version/s: (was: 2.0.0)

> Replication V2
> --
>
> Key: HBASE-14379
> URL: https://issues.apache.org/jira/browse/HBASE-14379
> Project: HBase
>  Issue Type: Umbrella
>  Components: Replication
>Reporter: Andrew Purtell
>
> Replication V2 is a tear-down of exiting replication code to just the 
> interfaces introduced in HBASE-11367, then a rebuild around the following 
> principles, goals, and suggested features:
> - No state in ZooKeeper. Introduce a new system table for tracking peers, 
> queues, and log positions. (Some discussion on HBASE-10295, probably will be 
> replaced with a set of more focused issues.)
> - Allow replication v1 and v2 to coexist. Note all of the undesirable 
> features of v1 will remain as long as v1 is active, 'fixing' v1 is out of 
> scope. Supporting communication between v1 and v2 endpoints would also be out 
> of scope.
> - Simplified internal programming model based on iterators
> - Streaming data transfer
> - Administrative actions mediated by the master with support for security 
> hooks (like HBASE-11392)
> - Replication state persisted and communicated with protobuf (like 
> HBASE-11393 but everywhere)
> - Detailed metrics
> - Support for at least simple status checks and admin actions via UI and shell
> - Hbck support for fixing corrupt or stuck queues (like HBASE-14014)
> - Support for bulk load, perhaps through augmenting bulk load to build WALs 
> as well as HFiles (see HBASE-13153)
> - Optional consideration for replicating schema as well as data (like 
> HBASE-12947). May fall out of scope.
> - Optional separation of replication function from the regionservers (see 
> HBASE-8772)
> - Optional alternate scheduling of edits besides FIFO-by-region (see 
> HBASE-1734 and HBASE-14014)
> There are a number of existing JIRAs that will eventually be closed as 
> duplicate, wont fix, or reparented here.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18255) Time-Delayed HBase Performance Degradation with Java 7

2017-07-05 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075596#comment-16075596
 ] 

Hudson commented on HBASE-18255:


SUCCESS: Integrated in Jenkins build HBase-1.1-JDK8 #1972 (See 
[https://builds.apache.org/job/HBase-1.1-JDK8/1972/])
HBASE-18255 Set JVM code cache size in provided configuration (Vladimir 
(elserj: rev 975eaabad0f90035bfab17be6606ca56359a97bf)
* (edit) conf/hbase-env.cmd
* (edit) conf/hbase-env.sh


> Time-Delayed HBase Performance Degradation with Java 7
> --
>
> Key: HBASE-18255
> URL: https://issues.apache.org/jira/browse/HBASE-18255
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.1, 1.2.6, 1.1.11
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 1.4.0, 1.3.2, 1.2.7, 1.1.12
>
> Attachments: HBASE-18255-branch-1.x.v1.patch
>
>
> The good summary of the issue and provided resolution can be found in this 
> article:
> https://community.hortonworks.com/articles/105802/time-delayed-hbase-performance-degradation-with-ja.html
> In a few words, due to internal JVM 7 bug (which has been addressed only in 
> Java 8), HotSpot code cache can become full and after that ALL JIT 
> compilations get suspended indefinitely.  The default value for code cache 
> size in JVM 7 is quite low: 48MB. It is recommended to increase this value at 
> least to 256MB (default in JVM 8).
> This BUG affects only 1.x 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-10414) Distributed replay should re-encode WAL entries with its own RPC codec

2017-07-05 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-10414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075598#comment-16075598
 ] 

stack commented on HBASE-10414:
---

Unscheduling old issue not being worked on.

> Distributed replay should re-encode WAL entries with its own RPC codec
> --
>
> Key: HBASE-10414
> URL: https://issues.apache.org/jira/browse/HBASE-10414
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 0.98.0
>Reporter: Andrew Purtell
>
> HBASE-10412 allows distributed replay to send WAL entries with tags intact 
> between RegionServers by substituting a WALCodec directly for the RPC codec. 
> We should instead have distributed replay handle WAL entries including tags 
> with its own tag-aware RPC codec and drop the direct use of WALCodecs for 
> that purpose. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-10414) Distributed replay should re-encode WAL entries with its own RPC codec

2017-07-05 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-10414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-10414:
--
Fix Version/s: (was: 2.0.0)

> Distributed replay should re-encode WAL entries with its own RPC codec
> --
>
> Key: HBASE-10414
> URL: https://issues.apache.org/jira/browse/HBASE-10414
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 0.98.0
>Reporter: Andrew Purtell
>
> HBASE-10412 allows distributed replay to send WAL entries with tags intact 
> between RegionServers by substituting a WALCodec directly for the RPC codec. 
> We should instead have distributed replay handle WAL entries including tags 
> with its own tag-aware RPC codec and drop the direct use of WALCodecs for 
> that purpose. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18108) Procedure WALs are archived but not cleaned; fix

2017-07-05 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075591#comment-16075591
 ] 

stack commented on HBASE-18108:
---

So, we already have a WAL cleaner for the general RegionServer WALs. Can we 
adapt this WAL cleaner to also harvest old Pv2 Master WALs? It would be a pity 
running yet another chore/thread in master -- one for RegionServer WALs and 
then another for the master procedure WALs?

As the master runs it creates procedure WALs. When the procedure system is done 
w/ them, it moves them to an archive dir. This archive dir was a hack up so 
feel free to rename or relocate it.  The idea is that after some configurable 
amount of time had passed, then the master procedure wals in the archive would 
be deleted... as they are for the regionserver WALs.

> Procedure WALs are archived but not cleaned; fix
> 
>
> Key: HBASE-18108
> URL: https://issues.apache.org/jira/browse/HBASE-18108
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Affects Versions: 2.0.0
>Reporter: stack
>Priority: Critical
> Fix For: 2.0.0
>
>
> The Procedure WAL files used to be deleted when done. HBASE-14614 keeps them 
> around in case issue but what is missing is a GC for no-longer-needed WAL 
> files. This one is pretty important.
> From WALProcedureStore Cleaner TODO in 
> https://docs.google.com/document/d/1eVKa7FHdeoJ1-9o8yZcOTAQbv0u0bblBlCCzVSIn69g/edit#heading=h.r2pc835nb7vi



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18106) Redo ProcedureInfo and LockInfo

2017-07-05 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075586#comment-16075586
 ] 

stack commented on HBASE-18106:
---

Idea is to purge ProcedureInfo from our API. You'd have to deprecate it since 
it has been in API since before 2.0.

Add a method that returns an array of Strings, JSON Strings. Deprecate 
listProcedures in favor of getProcedures which returns String [] and ditto for 
listLocks and getLocks.

Unfortunately we have internally made use of ProcedureInfo to check owner.  
Would have to come up w/ something else or parse the json to find the owner 
key/value and compare that.

Serializing as JSON, you'd just serialize the protobuf. You'd probably have to 
filter out the serialized byte array of procedure data.

Use the protobuf-util which is available via 
hbase-thirdparty/hbase-shaded-miscellaneous. See comment higher up on how you 
might use it.


> Redo ProcedureInfo and LockInfo
> ---
>
> Key: HBASE-18106
> URL: https://issues.apache.org/jira/browse/HBASE-18106
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Affects Versions: 2.0.0
>Reporter: stack
>Priority: Critical
> Fix For: 2.0.0
>
>
> ProcedureInfo was introduced as a lowest-common-denominator POJO that could 
> be used as a facade on PB Procedures. It was good for showing state of 
> Procedure framework in shell and UI.
> Its a bit weird though. Its up in hbase-common rather than in Procedure and 
> it can only ever show a subset of the Procedure info.
> I was thinking we could use the pb3.1 pb->JSON utility instead and emit a 
> JSON String wherever we need to export a view on procedure internals.
> This issue is about exploring this possibility. Would depend on our having an 
> upgraded guava (so probably depends on the 'pre-build' project).
> From ProcedureInfo and LockInfo need fixing in 
> https://docs.google.com/document/d/1eVKa7FHdeoJ1-9o8yZcOTAQbv0u0bblBlCCzVSIn69g/edit#heading=h.kid1jzo114xw



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18324) [hbase-thirdparty] Tooling to prevent commits that mistakenly make use of unshaded protobuf/guava/etc.

2017-07-05 Thread Mike Drob (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075585#comment-16075585
 ] 

Mike Drob commented on HBASE-18324:
---

[~apurtell] - that looks good. do we still have to implement custom rules for 
it? I didn't see anything in the default ruleset to ban certain imports.

> [hbase-thirdparty] Tooling to prevent commits that mistakenly make use of 
> unshaded protobuf/guava/etc.
> --
>
> Key: HBASE-18324
> URL: https://issues.apache.org/jira/browse/HBASE-18324
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>
> Chatting w/ [~mdrob], he brought up the question of what if a dev makes use 
> of unshaded protobuf or guava? Afterall, the old jars (pb2.5 and hadoops 
> guava12.0 or whatever) are still on the CLASSPATH because upstream depends on 
> them.
> We could add a check as part of prebuild. It would complain if use of 
> com.google instead of org.apache.hadoop.hbase.shaded.com.google. But even 
> then, there are cases where com.google is legit (coprocessor endpoints) so it 
> would have to let these pass.
> TODO.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18106) Redo ProcedureInfo and LockInfo

2017-07-05 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075571#comment-16075571
 ] 

stack commented on HBASE-18106:
---

Perhaps graphs could be done with a d3 lib if we had an API that gave out JSON 
versions of procedure info and lock info.

> Redo ProcedureInfo and LockInfo
> ---
>
> Key: HBASE-18106
> URL: https://issues.apache.org/jira/browse/HBASE-18106
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Affects Versions: 2.0.0
>Reporter: stack
>Priority: Critical
> Fix For: 2.0.0
>
>
> ProcedureInfo was introduced as a lowest-common-denominator POJO that could 
> be used as a facade on PB Procedures. It was good for showing state of 
> Procedure framework in shell and UI.
> Its a bit weird though. Its up in hbase-common rather than in Procedure and 
> it can only ever show a subset of the Procedure info.
> I was thinking we could use the pb3.1 pb->JSON utility instead and emit a 
> JSON String wherever we need to export a view on procedure internals.
> This issue is about exploring this possibility. Would depend on our having an 
> upgraded guava (so probably depends on the 'pre-build' project).
> From ProcedureInfo and LockInfo need fixing in 
> https://docs.google.com/document/d/1eVKa7FHdeoJ1-9o8yZcOTAQbv0u0bblBlCCzVSIn69g/edit#heading=h.kid1jzo114xw



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17705) Procedure execution must fail fast if procedure is not registered

2017-07-05 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075568#comment-16075568
 ] 

stack commented on HBASE-17705:
---

IOE is our base exception so catching it is a little indiscriminating. Can the 
test have more precision? Catch DoNotRetryIOE at least or some such? Otherwise, 
nice test and patch looks good.

> Procedure execution must fail fast if procedure is not registered
> -
>
> Key: HBASE-17705
> URL: https://issues.apache.org/jira/browse/HBASE-17705
> Project: HBase
>  Issue Type: Bug
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: HBASE-17705.v2.patch
>
>
> The issue was discovered during backup testing and described here:
> https://issues.apache.org/jira/browse/HBASE-14123?focusedCommentId=15886712=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15886712
> If procedure is not registered, client will be retrying until max attempts 
> reached. The Master should throw DoNotRetryIOException and client RPC should 
> handle this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17908) Upgrade guava

2017-07-05 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075563#comment-16075563
 ] 

stack commented on HBASE-17908:
---

.0018 Rebase. When build ran, it could not find 017 (temporary JIRA outage) so 
it applied 016 which does not apply... Retry.

> Upgrade guava
> -
>
> Key: HBASE-17908
> URL: https://issues.apache.org/jira/browse/HBASE-17908
> Project: HBase
>  Issue Type: Sub-task
>  Components: dependencies
>Reporter: Balazs Meszaros
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-17908.master.001.patch, 
> HBASE-17908.master.002.patch, HBASE-17908.master.003.patch, 
> HBASE-17908.master.004.patch, HBASE-17908.master.005.patch, 
> HBASE-17908.master.006.patch, HBASE-17908.master.007.patch, 
> HBASE-17908.master.008.patch, HBASE-17908.master.009.patch, 
> HBASE-17908.master.010.patch, HBASE-17908.master.011.patch, 
> HBASE-17908.master.012.patch, HBASE-17908.master.013.patch, 
> HBASE-17908.master.013.patch, HBASE-17908.master.014.patch, 
> HBASE-17908.master.015.patch, HBASE-17908.master.015.patch, 
> HBASE-17908.master.016.patch, HBASE-17908.master.017.patch, 
> HBASE-17908.master.018.patch
>
>
> Currently we are using guava 12.0.1, but the latest version is 21.0. 
> Upgrading guava is always a hassle because it is not always backward 
> compatible with itself.
> Currently I think there are to approaches:
> 1. Upgrade guava to the newest version (21.0) and shade it.
> 2. Upgrade guava to a version which does not break or builds (15.0).
> If we can update it, some dependencies should be removed: 
> commons-collections, commons-codec, ...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-12349) Add Maven build support module for a custom version of error-prone

2017-07-05 Thread Mike Drob (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-12349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075564#comment-16075564
 ] 

Mike Drob commented on HBASE-12349:
---

This might be easier now:

{quote}
Maven
Starting in version 3.5, maven-compiler-plugin allows the processor path to be 
configured with the annotationProcessorPaths parameter.

For a complete example, see: 
[examples/plugin/maven|https://github.com/google/error-prone/tree/master/examples/plugin/maven].
{quote}

So I think we could add the additional bug checks without needing to fork or 
publish a custom error-prone version.

> Add Maven build support module for a custom version of error-prone
> --
>
> Key: HBASE-12349
> URL: https://issues.apache.org/jira/browse/HBASE-12349
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Andrew Purtell
> Fix For: 2.0.0
>
>
> Add a new Maven build support module that builds and publishes a custom 
> error-prone artifact for use by the rest of the build.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-17908) Upgrade guava

2017-07-05 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-17908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-17908:
--
Attachment: HBASE-17908.master.018.patch

> Upgrade guava
> -
>
> Key: HBASE-17908
> URL: https://issues.apache.org/jira/browse/HBASE-17908
> Project: HBase
>  Issue Type: Sub-task
>  Components: dependencies
>Reporter: Balazs Meszaros
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-17908.master.001.patch, 
> HBASE-17908.master.002.patch, HBASE-17908.master.003.patch, 
> HBASE-17908.master.004.patch, HBASE-17908.master.005.patch, 
> HBASE-17908.master.006.patch, HBASE-17908.master.007.patch, 
> HBASE-17908.master.008.patch, HBASE-17908.master.009.patch, 
> HBASE-17908.master.010.patch, HBASE-17908.master.011.patch, 
> HBASE-17908.master.012.patch, HBASE-17908.master.013.patch, 
> HBASE-17908.master.013.patch, HBASE-17908.master.014.patch, 
> HBASE-17908.master.015.patch, HBASE-17908.master.015.patch, 
> HBASE-17908.master.016.patch, HBASE-17908.master.017.patch, 
> HBASE-17908.master.018.patch
>
>
> Currently we are using guava 12.0.1, but the latest version is 21.0. 
> Upgrading guava is always a hassle because it is not always backward 
> compatible with itself.
> Currently I think there are to approaches:
> 1. Upgrade guava to the newest version (21.0) and shade it.
> 2. Upgrade guava to a version which does not break or builds (15.0).
> If we can update it, some dependencies should be removed: 
> commons-collections, commons-codec, ...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-17056) Remove checked in PB generated files

2017-07-05 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-17056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-17056:
--
Attachment: HBASE-17056.master.003.patch

> Remove checked in PB generated files 
> -
>
> Key: HBASE-17056
> URL: https://issues.apache.org/jira/browse/HBASE-17056
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Enis Soztutar
>Assignee: stack
> Fix For: 2.0.0
>
> Attachments: 
> 0002-HBASE-17056-Remove-checked-in-PB-generated-files.patch, 
> HBASE-17056.master.001.patch, HBASE-17056.master.002.patch, 
> HBASE-17056.master.003.patch
>
>
> Now that we have the new PB maven plugin, there is no need to have the PB 
> files checked in to the repo. The reason we did that was to ease up developer 
> env setup. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17056) Remove checked in PB generated files

2017-07-05 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075552#comment-16075552
 ] 

stack commented on HBASE-17056:
---

.003 rebase.

> Remove checked in PB generated files 
> -
>
> Key: HBASE-17056
> URL: https://issues.apache.org/jira/browse/HBASE-17056
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Enis Soztutar
>Assignee: stack
> Fix For: 2.0.0
>
> Attachments: 
> 0002-HBASE-17056-Remove-checked-in-PB-generated-files.patch, 
> HBASE-17056.master.001.patch, HBASE-17056.master.002.patch, 
> HBASE-17056.master.003.patch
>
>
> Now that we have the new PB maven plugin, there is no need to have the PB 
> files checked in to the repo. The reason we did that was to ease up developer 
> env setup. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18324) [hbase-thirdparty] Tooling to prevent commits that mistakenly make use of unshaded protobuf/guava/etc.

2017-07-05 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075549#comment-16075549
 ] 

stack commented on HBASE-18324:
---

Oh. Sweet. Something to work with. On not having excludes, it might be ok since 
if you are referring to com.google.protobuf, then you need to be in the 
hbase-endpoint module... or in your own purposed module... i.e. outside of the 
hbase-server, et al.  Let me play w/ this and errorprone see if I can come 
up w/ something. Thanks lads.

> [hbase-thirdparty] Tooling to prevent commits that mistakenly make use of 
> unshaded protobuf/guava/etc.
> --
>
> Key: HBASE-18324
> URL: https://issues.apache.org/jira/browse/HBASE-18324
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>
> Chatting w/ [~mdrob], he brought up the question of what if a dev makes use 
> of unshaded protobuf or guava? Afterall, the old jars (pb2.5 and hadoops 
> guava12.0 or whatever) are still on the CLASSPATH because upstream depends on 
> them.
> We could add a check as part of prebuild. It would complain if use of 
> com.google instead of org.apache.hadoop.hbase.shaded.com.google. But even 
> then, there are cases where com.google is legit (coprocessor endpoints) so it 
> would have to let these pass.
> TODO.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18107) [AMv2] Rename DispatchMergingRegionsRequest & DispatchMergingRegions

2017-07-05 Thread Yi Liang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075546#comment-16075546
 ] 

Yi Liang commented on HBASE-18107:
--

Hi [~stack], below is my finds about this DispatchMergingRegions
here is the 2 execute paths of merge in two branches
{quote}
In branch-1.2:
HBaseAdmin.mergeRegions 
-> MasterKeepAliveConnection.dispatchMergingRegions
-> MasterRpcServices.dispatchMergingRegions
-> HMaster.dispatchMergingRegions
-> DispatchMergingRegionHandler
In branch master:
HBaseAdmin.mergeRegionsAsync 
-> MasterKeepAliveConnection.mergeTableRegions
-> MasterRpcServices.mergeTableRegions
-> HMaster.mergeRegions 
-> MasterProcedureUtil.submitProcedure 
{quote}

It seems that [~syuanjiang]  have replaced  dispatchMergingRegions with 
MergeTableRegionsProcedure, followed by the history of merge procedure:

In HBASE-14552,The DispatchMergingRegionHandler has been replaced with 
DispatchMergingRegionsProcedure

In HBASE-16119 DispatchMergingRegionsProcedure has been replaced with 
MergeTableRegionsProcedure

In HBASE-17470 DispatchMergingRegionsProcedure has been removed

In HBase-14614, it seems that DispatchMergingRegionsProcedure added back again


So from my point of view, we need to remove DispatchMergingRegionsProcedure
i.e we need to remove the path 2 of Merge Regions as I mentioned in HBASE-18105




> [AMv2] Rename DispatchMergingRegionsRequest & DispatchMergingRegions
> 
>
> Key: HBASE-18107
> URL: https://issues.apache.org/jira/browse/HBASE-18107
> Project: HBase
>  Issue Type: Sub-task
>  Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: stack
> Fix For: 2.0.0
>
>
> They don't align with how we have named the Split equivalents; i.e. 
> SplitRegion (so should be MergeRegion...). They probably have these awkward 
> names because the obvious slots are occupied... so this may not be fixable 
> but filing issue anyways.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18324) [hbase-thirdparty] Tooling to prevent commits that mistakenly make use of unshaded protobuf/guava/etc.

2017-07-05 Thread Andrew Purtell (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075544#comment-16075544
 ] 

Andrew Purtell commented on HBASE-18324:


We have HBASE-11912 FWIW

> [hbase-thirdparty] Tooling to prevent commits that mistakenly make use of 
> unshaded protobuf/guava/etc.
> --
>
> Key: HBASE-18324
> URL: https://issues.apache.org/jira/browse/HBASE-18324
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>
> Chatting w/ [~mdrob], he brought up the question of what if a dev makes use 
> of unshaded protobuf or guava? Afterall, the old jars (pb2.5 and hadoops 
> guava12.0 or whatever) are still on the CLASSPATH because upstream depends on 
> them.
> We could add a check as part of prebuild. It would complain if use of 
> com.google instead of org.apache.hadoop.hbase.shaded.com.google. But even 
> then, there are cases where com.google is legit (coprocessor endpoints) so it 
> would have to let these pass.
> TODO.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17908) Upgrade guava

2017-07-05 Thread Jerry He (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075539#comment-16075539
 ] 

Jerry He commented on HBASE-17908:
--

bq. All this stuff gets expunged by the concurrent removal of all checked-in 
generated files, HBASE-17056.

Ok.

> Upgrade guava
> -
>
> Key: HBASE-17908
> URL: https://issues.apache.org/jira/browse/HBASE-17908
> Project: HBase
>  Issue Type: Sub-task
>  Components: dependencies
>Reporter: Balazs Meszaros
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-17908.master.001.patch, 
> HBASE-17908.master.002.patch, HBASE-17908.master.003.patch, 
> HBASE-17908.master.004.patch, HBASE-17908.master.005.patch, 
> HBASE-17908.master.006.patch, HBASE-17908.master.007.patch, 
> HBASE-17908.master.008.patch, HBASE-17908.master.009.patch, 
> HBASE-17908.master.010.patch, HBASE-17908.master.011.patch, 
> HBASE-17908.master.012.patch, HBASE-17908.master.013.patch, 
> HBASE-17908.master.013.patch, HBASE-17908.master.014.patch, 
> HBASE-17908.master.015.patch, HBASE-17908.master.015.patch, 
> HBASE-17908.master.016.patch, HBASE-17908.master.017.patch
>
>
> Currently we are using guava 12.0.1, but the latest version is 21.0. 
> Upgrading guava is always a hassle because it is not always backward 
> compatible with itself.
> Currently I think there are to approaches:
> 1. Upgrade guava to the newest version (21.0) and shade it.
> 2. Upgrade guava to a version which does not break or builds (15.0).
> If we can update it, some dependencies should be removed: 
> commons-collections, commons-codec, ...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18324) [hbase-thirdparty] Tooling to prevent commits that mistakenly make use of unshaded protobuf/guava/etc.

2017-07-05 Thread Mike Drob (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075526#comment-16075526
 ] 

Mike Drob commented on HBASE-18324:
---

Maybe https://github.com/skuzzle/restrict-imports-enforcer-rule can help here. 
I don't see a good way to use that to allow specific exceptions in, though.

> [hbase-thirdparty] Tooling to prevent commits that mistakenly make use of 
> unshaded protobuf/guava/etc.
> --
>
> Key: HBASE-18324
> URL: https://issues.apache.org/jira/browse/HBASE-18324
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>
> Chatting w/ [~mdrob], he brought up the question of what if a dev makes use 
> of unshaded protobuf or guava? Afterall, the old jars (pb2.5 and hadoops 
> guava12.0 or whatever) are still on the CLASSPATH because upstream depends on 
> them.
> We could add a check as part of prebuild. It would complain if use of 
> com.google instead of org.apache.hadoop.hbase.shaded.com.google. But even 
> then, there are cases where com.google is legit (coprocessor endpoints) so it 
> would have to let these pass.
> TODO.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18324) [hbase-thirdparty] Tooling to prevent commits that mistakenly make use of unshaded protobuf/guava/etc.

2017-07-05 Thread stack (JIRA)

stack created HBASE-18324:
-

 Summary: [hbase-thirdparty] Tooling to prevent commits that 
mistakenly make use of unshaded protobuf/guava/etc.
 Key: HBASE-18324
 URL: https://issues.apache.org/jira/browse/HBASE-18324
 Project: HBase
  Issue Type: Bug
Reporter: stack


Chatting w/ [~mdrob], he brought up the question of what if a dev makes use of 
unshaded protobuf or guava? Afterall, the old jars (pb2.5 and hadoops guava12.0 
or whatever) are still on the CLASSPATH because upstream depends on them.

We could add a check as part of prebuild. It would complain if use of 
com.google instead of org.apache.hadoop.hbase.shaded.com.google. But even then, 
there are cases where com.google is legit (coprocessor endpoints) so it would 
have to let these pass.

TODO.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17056) Remove checked in PB generated files

2017-07-05 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075514#comment-16075514
 ] 

stack commented on HBASE-17056:
---

bq. This is no longer accurate, yes? Same applied to hbase-examples.

It is in that now the files are generated inline with the build. Before you had 
to generate them via a special step done pre-build.

On the poms, yeah, sorry, they were wonky so did a format. Pom changes are just 
adding dependency on hbase-thirdparty and narrowing guava includes by adding 
excludes to hadoop-* artifacts.

Thanks for the review [~mdrob]

Let me get a clean run via hadoop qa.

> Remove checked in PB generated files 
> -
>
> Key: HBASE-17056
> URL: https://issues.apache.org/jira/browse/HBASE-17056
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Enis Soztutar
>Assignee: stack
> Fix For: 2.0.0
>
> Attachments: 
> 0002-HBASE-17056-Remove-checked-in-PB-generated-files.patch, 
> HBASE-17056.master.001.patch, HBASE-17056.master.002.patch
>
>
> Now that we have the new PB maven plugin, there is no need to have the PB 
> files checked in to the repo. The reason we did that was to ease up developer 
> env setup. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17201) Edit of HFileBlock comments and javadoc

2017-07-05 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075481#comment-16075481
 ] 

Hudson commented on HBASE-17201:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3319 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/3319/])
HBASE-17201 Edit of HFileBlock comments and javadoc (stack: rev 
b71509151e6b079486f9ac76762279d70f1aa0ee)
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java


> Edit of HFileBlock comments and javadoc
> ---
>
> Key: HBASE-17201
> URL: https://issues.apache.org/jira/browse/HBASE-17201
> Project: HBase
>  Issue Type: Sub-task
>  Components: documentation
>Affects Versions: 2.0.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0
>
> Attachments: HBASE-17201.master.001.patch, 
> HBASE-17201.master.002.patch, HBASE-17201.master.002.patch
>
>
> Spent time in HFileBlock trying to do the parent issue. Failed. But did a 
> bunch of edits of comments/javadoc. Let me get those in at least.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-16120) Add shell test for truncate_preserve

2017-07-05 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-16120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075480#comment-16075480
 ] 

Hudson commented on HBASE-16120:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3319 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/3319/])
HBASE-16120 Add shell test for truncate_preserve (stack: rev 
619d6a50f6e0037d017fbac88274e3e85643cbf3)
* (edit) hbase-shell/src/test/ruby/hbase/admin_test.rb


> Add shell test for truncate_preserve
> 
>
> Key: HBASE-16120
> URL: https://issues.apache.org/jira/browse/HBASE-16120
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0, 1.4.0
>Reporter: Appy
>Assignee: Guangxu Cheng
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16120-master-v0.patch, 
> HBASE-16120-master-v0.patch, HBASE-16120-master-v0.patch
>
>
> There's no shell test for truncate_preserve, should add one.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-16730) Exclude junit as a transitive dependency from hadoop-common

2017-07-05 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-16730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075479#comment-16075479
 ] 

Hudson commented on HBASE-16730:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3319 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/3319/])
HBASE-16730 Excluded junit as a transitive dependency from hadoop-common 
(stack: rev 24435f65a5160bca773fa5ac61947bc3880ff368)
* (edit) pom.xml
* (edit) hbase-spark/pom.xml


> Exclude junit as a transitive dependency from hadoop-common
> ---
>
> Key: HBASE-16730
> URL: https://issues.apache.org/jira/browse/HBASE-16730
> Project: HBase
>  Issue Type: Improvement
>  Components: hbase
>Reporter: Nils Larsgård
>Assignee: Jan Hentschel
>Priority: Trivial
>  Labels: hbase-client, junit
> Fix For: 2.0.0
>
> Attachments: HBASE-16730.master.001.patch, HBASE-16730.master.002 
> (1).patch, HBASE-16730.master.002.patch, HBASE-16730.master.002.patch, 
> HBASE-16730.master.002.patch
>
>   Original Estimate: 20m
>  Remaining Estimate: 20m
>
> add exclusion to the hadoop-common dependency in hbase-client: exclude junit



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-17705) Procedure execution must fail fast if procedure is not registered

2017-07-05 Thread Vladimir Rodionov (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-17705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-17705:
--
Status: Patch Available  (was: Open)

> Procedure execution must fail fast if procedure is not registered
> -
>
> Key: HBASE-17705
> URL: https://issues.apache.org/jira/browse/HBASE-17705
> Project: HBase
>  Issue Type: Bug
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: HBASE-17705.v2.patch
>
>
> The issue was discovered during backup testing and described here:
> https://issues.apache.org/jira/browse/HBASE-14123?focusedCommentId=15886712=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15886712
> If procedure is not registered, client will be retrying until max attempts 
> reached. The Master should throw DoNotRetryIOException and client RPC should 
> handle this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-17705) Procedure execution must fail fast if procedure is not registered

2017-07-05 Thread Vladimir Rodionov (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-17705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-17705:
--
Status: Open  (was: Patch Available)

> Procedure execution must fail fast if procedure is not registered
> -
>
> Key: HBASE-17705
> URL: https://issues.apache.org/jira/browse/HBASE-17705
> Project: HBase
>  Issue Type: Bug
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: HBASE-17705.v2.patch
>
>
> The issue was discovered during backup testing and described here:
> https://issues.apache.org/jira/browse/HBASE-14123?focusedCommentId=15886712=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15886712
> If procedure is not registered, client will be retrying until max attempts 
> reached. The Master should throw DoNotRetryIOException and client RPC should 
> handle this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-17705) Procedure execution must fail fast if procedure is not registered

2017-07-05 Thread Vladimir Rodionov (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-17705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-17705:
--
Attachment: (was: HBASE-17705.v1.patch)

> Procedure execution must fail fast if procedure is not registered
> -
>
> Key: HBASE-17705
> URL: https://issues.apache.org/jira/browse/HBASE-17705
> Project: HBase
>  Issue Type: Bug
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: HBASE-17705.v2.patch
>
>
> The issue was discovered during backup testing and described here:
> https://issues.apache.org/jira/browse/HBASE-14123?focusedCommentId=15886712=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15886712
> If procedure is not registered, client will be retrying until max attempts 
> reached. The Master should throw DoNotRetryIOException and client RPC should 
> handle this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18255) Time-Delayed HBase Performance Degradation with Java 7

2017-07-05 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075461#comment-16075461
 ] 

Hudson commented on HBASE-18255:


SUCCESS: Integrated in Jenkins build HBase-1.2-IT #893 (See 
[https://builds.apache.org/job/HBase-1.2-IT/893/])
HBASE-18255 Set JVM code cache size in provided configuration (Vladimir 
(elserj: rev 8a996e3413e93ff7b421a9a5c611ad122439d681)
* (edit) conf/hbase-env.cmd
* (edit) conf/hbase-env.sh


> Time-Delayed HBase Performance Degradation with Java 7
> --
>
> Key: HBASE-18255
> URL: https://issues.apache.org/jira/browse/HBASE-18255
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.1, 1.2.6, 1.1.11
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 1.4.0, 1.3.2, 1.2.7, 1.1.12
>
> Attachments: HBASE-18255-branch-1.x.v1.patch
>
>
> The good summary of the issue and provided resolution can be found in this 
> article:
> https://community.hortonworks.com/articles/105802/time-delayed-hbase-performance-degradation-with-ja.html
> In a few words, due to internal JVM 7 bug (which has been addressed only in 
> Java 8), HotSpot code cache can become full and after that ALL JIT 
> compilations get suspended indefinitely.  The default value for code cache 
> size in JVM 7 is quite low: 48MB. It is recommended to increase this value at 
> least to 256MB (default in JVM 8).
> This BUG affects only 1.x 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17908) Upgrade guava

2017-07-05 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075458#comment-16075458
 ] 

stack commented on HBASE-17908:
---

Thank you for review [~jerryhe]

bq. At the end of this, we will still have the Guava (non-shaded, old version) 
as dependency, from hadoop-common?

Yes sir. We won't be using it other than indirectly.  Hadoop Configuration uses 
the guava precondition so it needs the old guava but our guava use is relocated 
so we'll not use hadoop's guava.  I spent a bit of effort shutting down all 
transitive includes of guavas so it only comes in w/ hadoop-common references.

bq. What are all the changes in hbase-protocol-shaded, generated and non 
generated?

Ripples from move from protobuf 3.2.0 to 3.3.0. I updated the protobuf we were 
referencing to match the protobuf that is brought in via hbase-thirdparty just 
in case the diff would make any issues.

All this stuff gets expunged by the concurrent removal of all checked-in 
generated files, HBASE-17056.

Thanks Jerry.



> Upgrade guava
> -
>
> Key: HBASE-17908
> URL: https://issues.apache.org/jira/browse/HBASE-17908
> Project: HBase
>  Issue Type: Sub-task
>  Components: dependencies
>Reporter: Balazs Meszaros
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-17908.master.001.patch, 
> HBASE-17908.master.002.patch, HBASE-17908.master.003.patch, 
> HBASE-17908.master.004.patch, HBASE-17908.master.005.patch, 
> HBASE-17908.master.006.patch, HBASE-17908.master.007.patch, 
> HBASE-17908.master.008.patch, HBASE-17908.master.009.patch, 
> HBASE-17908.master.010.patch, HBASE-17908.master.011.patch, 
> HBASE-17908.master.012.patch, HBASE-17908.master.013.patch, 
> HBASE-17908.master.013.patch, HBASE-17908.master.014.patch, 
> HBASE-17908.master.015.patch, HBASE-17908.master.015.patch, 
> HBASE-17908.master.016.patch, HBASE-17908.master.017.patch
>
>
> Currently we are using guava 12.0.1, but the latest version is 21.0. 
> Upgrading guava is always a hassle because it is not always backward 
> compatible with itself.
> Currently I think there are to approaches:
> 1. Upgrade guava to the newest version (21.0) and shade it.
> 2. Upgrade guava to a version which does not break or builds (15.0).
> If we can update it, some dependencies should be removed: 
> commons-collections, commons-codec, ...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17705) Procedure execution must fail fast if procedure is not registered

2017-07-05 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075455#comment-16075455
 ] 

Hadoop QA commented on HBASE-17705:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  2m 17s{color} 
| {color:red} HBASE-17705 does not apply to master. Rebase required? Wrong 
Branch? See https://yetus.apache.org/documentation/0.4.0/precommit-patchnames 
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-17705 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12855223/HBASE-17705.v1.patch |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7525/console |
| Powered by | Apache Yetus 0.4.0   http://yetus.apache.org |


This message was automatically generated.



> Procedure execution must fail fast if procedure is not registered
> -
>
> Key: HBASE-17705
> URL: https://issues.apache.org/jira/browse/HBASE-17705
> Project: HBase
>  Issue Type: Bug
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: HBASE-17705.v1.patch, HBASE-17705.v2.patch
>
>
> The issue was discovered during backup testing and described here:
> https://issues.apache.org/jira/browse/HBASE-14123?focusedCommentId=15886712=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15886712
> If procedure is not registered, client will be retrying until max attempts 
> reached. The Master should throw DoNotRetryIOException and client RPC should 
> handle this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17056) Remove checked in PB generated files

2017-07-05 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075456#comment-16075456
 ] 

Hadoop QA commented on HBASE-17056:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  3m 10s{color} 
| {color:red} HBASE-17056 does not apply to master. Rebase required? Wrong 
Branch? See https://yetus.apache.org/documentation/0.4.0/precommit-patchnames 
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-17056 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12875807/HBASE-17056.master.002.patch
 |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7524/console |
| Powered by | Apache Yetus 0.4.0   http://yetus.apache.org |


This message was automatically generated.



> Remove checked in PB generated files 
> -
>
> Key: HBASE-17056
> URL: https://issues.apache.org/jira/browse/HBASE-17056
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Enis Soztutar
>Assignee: stack
> Fix For: 2.0.0
>
> Attachments: 
> 0002-HBASE-17056-Remove-checked-in-PB-generated-files.patch, 
> HBASE-17056.master.001.patch, HBASE-17056.master.002.patch
>
>
> Now that we have the new PB maven plugin, there is no need to have the PB 
> files checked in to the repo. The reason we did that was to ease up developer 
> env setup. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17908) Upgrade guava

2017-07-05 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075451#comment-16075451
 ] 

Hadoop QA commented on HBASE-17908:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
27s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  0m 14s{color} 
| {color:red} HBASE-17908 does not apply to master. Rebase required? Wrong 
Branch? See https://yetus.apache.org/documentation/0.4.0/precommit-patchnames 
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:757bf37 |
| JIRA Issue | HBASE-17908 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12875692/HBASE-17908.master.016.patch
 |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7526/console |
| Powered by | Apache Yetus 0.4.0   http://yetus.apache.org |


This message was automatically generated.



> Upgrade guava
> -
>
> Key: HBASE-17908
> URL: https://issues.apache.org/jira/browse/HBASE-17908
> Project: HBase
>  Issue Type: Sub-task
>  Components: dependencies
>Reporter: Balazs Meszaros
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-17908.master.001.patch, 
> HBASE-17908.master.002.patch, HBASE-17908.master.003.patch, 
> HBASE-17908.master.004.patch, HBASE-17908.master.005.patch, 
> HBASE-17908.master.006.patch, HBASE-17908.master.007.patch, 
> HBASE-17908.master.008.patch, HBASE-17908.master.009.patch, 
> HBASE-17908.master.010.patch, HBASE-17908.master.011.patch, 
> HBASE-17908.master.012.patch, HBASE-17908.master.013.patch, 
> HBASE-17908.master.013.patch, HBASE-17908.master.014.patch, 
> HBASE-17908.master.015.patch, HBASE-17908.master.015.patch, 
> HBASE-17908.master.016.patch, HBASE-17908.master.017.patch
>
>
> Currently we are using guava 12.0.1, but the latest version is 21.0. 
> Upgrading guava is always a hassle because it is not always backward 
> compatible with itself.
> Currently I think there are to approaches:
> 1. Upgrade guava to the newest version (21.0) and shade it.
> 2. Upgrade guava to a version which does not break or builds (15.0).
> If we can update it, some dependencies should be removed: 
> commons-collections, commons-codec, ...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17908) Upgrade guava

2017-07-05 Thread Jerry He (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075444#comment-16075444
 ] 

Jerry He commented on HBASE-17908:
--

At the end of this, we will still have the Guava (non-shaded, old version) as 
dependency, from hadoop-common?
What are all the changes in hbase-protocol-shaded, generated and non generated? 

+1

> Upgrade guava
> -
>
> Key: HBASE-17908
> URL: https://issues.apache.org/jira/browse/HBASE-17908
> Project: HBase
>  Issue Type: Sub-task
>  Components: dependencies
>Reporter: Balazs Meszaros
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-17908.master.001.patch, 
> HBASE-17908.master.002.patch, HBASE-17908.master.003.patch, 
> HBASE-17908.master.004.patch, HBASE-17908.master.005.patch, 
> HBASE-17908.master.006.patch, HBASE-17908.master.007.patch, 
> HBASE-17908.master.008.patch, HBASE-17908.master.009.patch, 
> HBASE-17908.master.010.patch, HBASE-17908.master.011.patch, 
> HBASE-17908.master.012.patch, HBASE-17908.master.013.patch, 
> HBASE-17908.master.013.patch, HBASE-17908.master.014.patch, 
> HBASE-17908.master.015.patch, HBASE-17908.master.015.patch, 
> HBASE-17908.master.016.patch, HBASE-17908.master.017.patch
>
>
> Currently we are using guava 12.0.1, but the latest version is 21.0. 
> Upgrading guava is always a hassle because it is not always backward 
> compatible with itself.
> Currently I think there are to approaches:
> 1. Upgrade guava to the newest version (21.0) and shade it.
> 2. Upgrade guava to a version which does not break or builds (15.0).
> If we can update it, some dependencies should be removed: 
> commons-collections, commons-codec, ...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-15349) Update surefire version to 2.19.1

2017-07-05 Thread stack (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-15349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-15349.
---
Resolution: Implemented

Resolving as implemented. Was fixed by

author Peter Somogyi  Thu Jun 29 15:37:22 2017 +0200
committer Michael Stack  Mon Jul 3 19:43:42 2017 -0700

HBASE-18264 Update pom plugins

Update plugins in main and subprojects
Unified versions to use variable instead of direct values

> Update surefire version to 2.19.1
> -
>
> Key: HBASE-15349
> URL: https://issues.apache.org/jira/browse/HBASE-15349
> Project: HBase
>  Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Fix For: 2.0.0
>
> Attachments: HBASE-15349.patch
>
>
> So that new properties like surefire.excludesFile and includesFile can be 
> used to easily exclude/include flaky tests.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

1 2 3 >

1 - 100 of 258 matches

Mail list logo