[jira] [Created] (HBASE-19563) A few hbase-procedure classes missing @InterfaceAudience annotation

[Guanghao Zhang (JIRA)]

Guanghao Zhang created HBASE-19563:
--

 Summary: A few hbase-procedure classes missing @InterfaceAudience 
annotation
 Key: HBASE-19563
 URL: https://issues.apache.org/jira/browse/HBASE-19563
 Project: HBase
  Issue Type: Bug
  Components: proc-v2
Reporter: Guanghao Zhang
Priority: Minor


NoopProcedureStore.java
ProcedureStoreBase.java
ProcedureMetrics.java
LockStatus.java
LockAndQueue.java
ProcedureStateSerializer.java



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19562) Purge mirror writing of region and table info into fs at .tableinfo and .regioninfo

[Zach York (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297967#comment-16297967
 ] 

Zach York commented on HBASE-19562:
---

[~stack] I know you have some misgivings on the approach for HBASE-18840, but 
the current approach relies on reading regioninfo's from the filesystem to 
construct an initial meta table. If we are removing these files, is it possible 
to recover a meta table using just the storefiles (no meta table though)? I'd 
be happy to get rid of these files if there is a better way to do this :)

I'll also use this opportunity to give that JIRA a ping :)

> Purge mirror writing of region and table info into fs at .tableinfo and 
> .regioninfo
> ---
>
> Key: HBASE-19562
> URL: https://issues.apache.org/jira/browse/HBASE-19562
> Project: HBase
>  Issue Type: Bug
>  Components: fs
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-2
>
> Attachments: 
> 0002-HBASE-19562-Purge-mirror-writing-of-region-and-table.patch
>
>
> We don't use these files in hbase2 yet we keep writing them when we create a 
> table or region.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19553) Old replica regions should be cleared from AM memory after primary region split or merge

[Pankaj Kumar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297964#comment-16297964
 ] 

Pankaj Kumar commented on HBASE-19553:
--

Ok [~huaxiang], we can wait until HBASE-18625 committed, will upload the 
modified patch after that.

> Old replica regions should be cleared from AM memory after primary region 
> split or merge
> 
>
> Key: HBASE-19553
> URL: https://issues.apache.org/jira/browse/HBASE-19553
> Project: HBase
>  Issue Type: Bug
>  Components: master
>Reporter: huaxiang sun
>Assignee: Pankaj Kumar
>Priority: Minor
> Fix For: 1.5.0
>
> Attachments: HBASE-19553-branch-1-v2.patch, HBASE-19553-branch-1.patch
>
>
> Similar to HBASE-18025, the replica parent's info is not removed from master. 
> Actually I think it can be removed after replica region is split or merged, I 
> will check the logic and apply one patch.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-13428) Migration to hbase-2.0.0

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-13428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297963#comment-16297963
 ] 

stack commented on HBASE-13428:
---

Wrote data w/ 0.98 w/ ACL enabled. Added some grants. Stopped and started 
branch-2 over the 0.98 data. Seems to be the tail of hfile only. Running again 
w/ a closer eye on it

> Migration to hbase-2.0.0
> 
>
> Key: HBASE-13428
> URL: https://issues.apache.org/jira/browse/HBASE-13428
> Project: HBase
>  Issue Type: Umbrella
>  Components: migration
>Reporter: stack
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
>
> Opening a 2.0 umbrella migration issue. Lets hang off this one any tools and 
> expectations migrating from 1.0 (or earlier) to 2.0. So far there are none 
> that I know of though there is an expectation in HBASE-13373 that hfiles are 
> at least major version 2 and minor version 3.  Lets list all such 
> expectations, etc., here.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19562) Purge mirror writing of region and table info into fs at .tableinfo and .regioninfo

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297959#comment-16297959
 ] 

stack commented on HBASE-19562:
---

Should probably up the fs version number when we do this.

> Purge mirror writing of region and table info into fs at .tableinfo and 
> .regioninfo
> ---
>
> Key: HBASE-19562
> URL: https://issues.apache.org/jira/browse/HBASE-19562
> Project: HBase
>  Issue Type: Bug
>  Components: fs
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-2
>
> Attachments: 
> 0002-HBASE-19562-Purge-mirror-writing-of-region-and-table.patch
>
>
> We don't use these files in hbase2 yet we keep writing them when we create a 
> table or region.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19562) Purge mirror writing of region and table info into fs at .tableinfo and .regioninfo

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19562:
--
Attachment: 0002-HBASE-19562-Purge-mirror-writing-of-region-and-table.patch

Can remove a bunch of crud.

Patch is not finished. Need to come back to this.

> Purge mirror writing of region and table info into fs at .tableinfo and 
> .regioninfo
> ---
>
> Key: HBASE-19562
> URL: https://issues.apache.org/jira/browse/HBASE-19562
> Project: HBase
>  Issue Type: Bug
>  Components: fs
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-2
>
> Attachments: 
> 0002-HBASE-19562-Purge-mirror-writing-of-region-and-table.patch
>
>
> We don't use these files in hbase2 yet we keep writing them when we create a 
> table or region.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19491) Exclude flaky tests from nightly master run

[Appy (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297946#comment-16297946
 ] 

Appy commented on HBASE-19491:
--

For some reason, findbugs on root is not working, but running on individual 
modules is working. For time being, going with latter approach.
{noformat}
# When using MODULES=(.)


  findbugs detection: HBASE-19491




[Wed Dec 20 06:05:19 UTC 2017 INFO]: Personality: branch findbugs



{noformat}


{noformat}
# When using CHANGED_MODULES


  findbugs detection: HBASE-19491




[Wed Dec 20 02:48:26 UTC 2017 INFO]: Personality: branch findbugs
cd /testptch/hbase/hbase-build-support/hbase-error-prone
/usr/share/maven/bin/mvn test-compile findbugs:findbugs -DskipTests=true > 
/testptch/patchprocess/branch-findbugs-hbase-build-support_hbase-error-prone.txt
 2>&1
Elapsed:   0m 14s
cd /testptch/hbase/hbase-protocol-shaded
/usr/share/maven/bin/mvn test-compile findbugs:findbugs -DskipTests=true > 
/testptch/patchprocess/branch-findbugs-hbase-protocol-shaded.txt 2>&1
Elapsed:   1m 54s
cd /testptch/hbase/hbase-common
/usr/share/maven/bin/mvn test-compile findbugs:findbugs -DskipTests=true > 
/testptch/patchprocess/branch-findbugs-hbase-common.txt 2>&1
Elapsed:   0m 32s
cd /testptch/hbase/hbase-metrics-api
/usr/share/maven/bin/mvn test-compile findbugs:findbugs -DskipTests=true > 
/testptch/patchprocess/branch-findbugs-hbase-metrics-api.txt 2>&1
Elapsed:   0m 15s
cd /testptch/hbase/hbase-hadoop-compat
/usr/share/maven/bin/mvn test-compile findbugs:findbugs -DskipTests=true > 
/testptch/patchprocess/branch-findbugs-hbase-hadoop-compat.txt 2>&1
Elapsed:   0m 15s
cd /testptch/hbase/hbase-metrics
/usr/share/maven/bin/mvn test-compile findbugs:findbugs -DskipTests=true > 
/testptch/patchprocess/branch-findbugs-hbase-metrics.txt 2>&1
Elapsed:   0m 16s
cd /testptch/hbase/hbase-hadoop2-compat
/usr/share/maven/bin/mvn test-compile findbugs:findbugs -DskipTests=true > 
/testptch/patchprocess/branch-findbugs-hbase-hadoop2-compat.txt 2>&1
Elapsed:   0m 19s
cd /testptch/hbase/hbase-protocol
/usr/share/maven/bin/mvn test-compile findbugs:findbugs -DskipTests=true > 
/testptch/patchprocess/branch-findbugs-hbase-protocol.txt 2>&1
Elapsed:   1m  7s
cd /testptch/hbase/hbase-client
/usr/share/maven/bin/mvn test-compile findbugs:findbugs -DskipTests=true > 
/testptch/patchprocess/branch-findbugs-hbase-client.txt 2>&1
Elapsed:   0m 49s
cd /testptch/hbase/hbase-zookeeper
/usr/share/maven/bin/mvn test-compile findbugs:findbugs -DskipTests=true > 
/testptch/patchprocess/branch-findbugs-hbase-zookeeper.txt 2>&1
Elapsed:   0m 23s
cd /testptch/hbase/hbase-replication
/usr/share/maven/bin/mvn test-compile findbugs:findbugs -DskipTests=true > 
/testptch/patchprocess/branch-findbugs-hbase-replication.txt 2>&1
Elapsed:   0m 19s
cd /testptch/hbase/hbase-http
/usr/share/maven/bin/mvn test-compile findbugs:findbugs -DskipTests=true > 
/testptch/patchprocess/branch-findbugs-hbase-http.txt 2>&1
Elapsed:   0m 23s
cd /testptch/hbase/hbase-procedure
/usr/share/maven/bin/mvn test-compile findbugs:findbugs -DskipTests=true > 
/testptch/patchprocess/branch-findbugs-hbase-procedure.txt 2>&1
Elapsed:   0m 21s
cd /testptch/hbase/hbase-server
/usr/share/maven/bin/mvn test-compile findbugs:findbugs -DskipTests=true > 
/testptch/patchprocess/branch-findbugs-hbase-server.txt 2>&1
Elapsed:   1m 58s
cd /testptch/hbase/hbase-mapreduce
/usr/share/maven/bin/mvn test-compile findbugs:findbugs -DskipTests=true > 
/testptch/patchprocess/branch-findbugs-hbase-mapreduce.txt 2>&1
Elapsed:   0m 33s
cd /testptch/hbase/hbase-thrift
/usr/share/maven/bin/mvn test-compile findbugs:findbugs -DskipTests=true > 
/testptch/patchprocess/branch-findbugs-hbase-thrift.txt 2>&1
Elapsed:   0m 58s
cd /testptch/hbase/hbase-rsgroup
/usr/share/maven/bin/mvn test-compile findbugs:findbugs -DskipTests=true > 
/testptch/patchprocess/branch-findbugs-hbase-rsgroup.txt 2>&1
Elapsed:   0m 33s
cd /testptch/hbase/hbase-endpoint
/usr/share/maven/bin/mvn test-compile findbugs:findbugs -DskipTests=true > 
/testptch/patchprocess/branch-findbugs-hbase-endpoint.txt 2>&1
Elapsed:   0m 32s
cd /testptch/hbase/hbase-backup
/usr/share/maven/bin/mvn test-compile findbugs:findbugs 

[jira] [Created] (HBASE-19562) Purge mirror writing of region and table info into fs at .tableinfo and .regioninfo

[stack (JIRA)]

stack created HBASE-19562:
-

 Summary: Purge mirror writing of region and table info into fs at 
.tableinfo and .regioninfo
 Key: HBASE-19562
 URL: https://issues.apache.org/jira/browse/HBASE-19562
 Project: HBase
  Issue Type: Bug
  Components: fs
Reporter: stack
Assignee: stack
 Fix For: 2.0.0-beta-2


We don't use these files in hbase2 yet we keep writing them when we create a 
table or region.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19561) maxCacheSize in CacheEvictionStats can't be accumulated repeatedly When dealing with each region

[Guangxu Cheng (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guangxu Cheng updated HBASE-19561:
--
Status: Patch Available  (was: Open)

> maxCacheSize in CacheEvictionStats can't be accumulated repeatedly When 
> dealing with each region
> 
>
> Key: HBASE-19561
> URL: https://issues.apache.org/jira/browse/HBASE-19561
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
> Attachments: HBASE-19561.master.001.patch
>
>
> {code:title=HRegionServer.java|borderStyle=solid}
>   public CacheEvictionStats clearRegionBlockCache(Region region) {
> BlockCache blockCache = this.getCacheConfig().getBlockCache();
> long evictedBlocks = 0;
> for(Store store : region.getStores()) {
>   for(StoreFile hFile : store.getStorefiles()) {
> evictedBlocks += 
> blockCache.evictBlocksByHfileName(hFile.getPath().getName());
>   }
> }
> return CacheEvictionStats.builder()
> .withEvictedBlocks(evictedBlocks)
> .withMaxCacheSize(blockCache.getMaxSize())
> .build();
>   }
> {code}
> BlockCache is the regionserver level object.If we accumulate this value for 
> each region, we will get a wrong result.
> The value of maxCacheSize should be set to BlockCache.getMaxSize() after 
> processing all the regions on this regionserver.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19364) Truncate_preserve fails with table when replica region > 1

[Pankaj Kumar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297936#comment-16297936
 ] 

Pankaj Kumar commented on HBASE-19364:
--

Hi [~huaxiang], HBASE-17319 applicable for other branches (brnach-1.3/1.2) as 
well. Any plan to commit HBASE-17319 in other branches, this Jira is still open.

> Truncate_preserve fails with table when replica region > 1
> --
>
> Key: HBASE-19364
> URL: https://issues.apache.org/jira/browse/HBASE-19364
> Project: HBase
>  Issue Type: Bug
>  Components: master
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 1.5.0
>
> Attachments: HBASE-19364-branch-1.patch
>
>
> Root cause is same as HBASE-17319, here we need to exclude secondary regions 
> while reading meta.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19468) FNFE during scans and flushes

[ramkrishna.s.vasudevan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-19468:
---
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Pushed to branch-1, 1.4, branch-2 and master. For branch-1 and branch-1.4 
ensured the patch applies cleanly and there are no compile issues. Thanks for 
reporting this issue and appreciate all the careful reviews here. 

> FNFE during scans and flushes
> -
>
> Key: HBASE-19468
> URL: https://issues.apache.org/jira/browse/HBASE-19468
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver, Scanners
>Affects Versions: 1.3.1
>Reporter: Thiruvel Thirumoolan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 2.0.0, 1.4.1, 1.5.0, 1.3.3
>
> Attachments: HBASE-19468-poc.patch, HBASE-19468_1.4.patch, 
> HBASE-19468_master.patch
>
>
> We see FNFE exceptions on our 1.3 clusters when scans and flushes happen at 
> the same time. This causes regionserver to throw a UnknownScannerException 
> and client retries.
> This happens during the following sequence:
> 1. Scanner open, client fetched some rows from regionserver and working on it
> 2. Flush happens and storeScanner is updated with flushed files 
> (StoreScanner.updateReaders())
> 3. Compaction happens on the region while scanner is still open
> 4. compaction discharger runs and cleans up the newly flushed file as we 
> don't have new scanners on it yet.
> 5. Client issues scan.next and during StoreScanner.resetScannerStack(), we 
> get a FNFE. RegionServer throws a UnknownScannerThe client retries in 1.3. 
> With branch-1.4, the scan fails with a DoNotRetryIOException.
> [~ram_krish], My proposal is to increment the reader count during 
> updateReaders() and decrement it during resetScannerStack(), so discharger 
> doesn't clean it up. Scan lease expiries also have to be taken care of. Am I 
> missing anything? Is there a better approach?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (HBASE-19468) FNFE during scans and flushes

[ramkrishna.s.vasudevan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan reassigned HBASE-19468:
--

Assignee: ramkrishna.s.vasudevan

> FNFE during scans and flushes
> -
>
> Key: HBASE-19468
> URL: https://issues.apache.org/jira/browse/HBASE-19468
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver, Scanners
>Affects Versions: 1.3.1
>Reporter: Thiruvel Thirumoolan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 2.0.0, 1.4.1, 1.5.0, 1.3.3
>
> Attachments: HBASE-19468-poc.patch, HBASE-19468_1.4.patch, 
> HBASE-19468_master.patch
>
>
> We see FNFE exceptions on our 1.3 clusters when scans and flushes happen at 
> the same time. This causes regionserver to throw a UnknownScannerException 
> and client retries.
> This happens during the following sequence:
> 1. Scanner open, client fetched some rows from regionserver and working on it
> 2. Flush happens and storeScanner is updated with flushed files 
> (StoreScanner.updateReaders())
> 3. Compaction happens on the region while scanner is still open
> 4. compaction discharger runs and cleans up the newly flushed file as we 
> don't have new scanners on it yet.
> 5. Client issues scan.next and during StoreScanner.resetScannerStack(), we 
> get a FNFE. RegionServer throws a UnknownScannerThe client retries in 1.3. 
> With branch-1.4, the scan fails with a DoNotRetryIOException.
> [~ram_krish], My proposal is to increment the reader count during 
> updateReaders() and decrement it during resetScannerStack(), so discharger 
> doesn't clean it up. Scan lease expiries also have to be taken care of. Am I 
> missing anything? Is there a better approach?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19116) Currently the tail of hfiles with CellComparator* classname makes it so hbase1 can't open hbase2 written hfiles; fix

[ramkrishna.s.vasudevan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297916#comment-16297916
 ] 

ramkrishna.s.vasudevan commented on HBASE-19116:


That's silly.
Problem is here
{code}
else if 
(comparatorClassName.equals("org.apache.hadoop.hbase.KeyValue.RawBytesComparator")
{code}

It should have been
{code}
else if 
(comparatorClassName.equals("org.apache.hadoop.hbase.KeyValue$RawBytesComparator")
{code}
The '.' should be '$'. :(

> Currently the tail of hfiles with CellComparator* classname makes it so 
> hbase1 can't open hbase2 written hfiles; fix
> 
>
> Key: HBASE-19116
> URL: https://issues.apache.org/jira/browse/HBASE-19116
> Project: HBase
>  Issue Type: Sub-task
>  Components: HFile, migration
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> See tail of HBASE-19052 for discussion which concludes we should try and make 
> it so operators do not have to go to latest hbase version before they 
> upgrade, at least if we can avoid it.
> The necessary change of our default comparator from KV to Cell naming has 
> hfiles with tails that have the classname CellComparator in them in place of 
> KeyValueComparator. If an hbase1 tries to open them, it will fail not having 
> a CellComparator in its classpath (We have name of comparator in tail because 
> different files require different comparators... perhaps we write an alias 
> instead of a class one day... TODO). HBASE-16189 and HBASE-19052 are about 
> trying to carry knowledge of hbase2 back to hbase1, a brittle approach making 
> it so operators will have to upgrade to the latest branch-1 before they can 
> go to hbase2.
> This issue is about undoing our writing of an incompatible (to hbase1) tail, 
> not unless we really have to (and it sounds like we could do without writing 
> an incompatible tail) to see if we can avoid requiring operators go to 
> lastest branch-1 (we may end up needing this but lets a have a really good 
> reason for it if we do).
> Oh, let this filing be an answer to our [~anoop.hbase]'s old high-level 
> question over in HBASE-16189:
> bq. ...means when rolling upgrade done to 2.0, first users have to upgrade to 
> some 1.x versions which is having this fix and then to 2.0.. What do you guys 
> think Whether we should avoid this kind of indirection? cc Enis Soztutar, 
> Stack, Ted Yu, Matteo Bertozzi
> Yeah, lets try to avoid this if we can...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19483) Add proper privilege check for rsgroup commands

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297917#comment-16297917
 ] 

Hadoop QA commented on HBASE-19483:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  5m 
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  3m 
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  8m 
22s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  3m 
46s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  5m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  5m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
 6s{color} | {color:green} hbase-server: The patch generated 0 new + 81 
unchanged - 11 fixed = 81 total (was 92) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
12s{color} | {color:green} The patch hbase-rsgroup passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
21s{color} | {color:green} root: The patch generated 0 new + 100 unchanged - 11 
fixed = 100 total (was 111) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
44s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
19m 41s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  3m 
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}140m 38s{color} 
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
21s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}202m 37s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestEncryptionKeyRotation |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19483 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12902961/HBASE-19483.master.005.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 924a24f46550 3.13.0-133-generic 

[jira] [Updated] (HBASE-19148) Reevaluate default values of configurations

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19148:
--
Attachment: HBASE-19148.master.003.patch

Retry

> Reevaluate default values of configurations
> ---
>
> Key: HBASE-19148
> URL: https://issues.apache.org/jira/browse/HBASE-19148
> Project: HBase
>  Issue Type: Bug
>  Components: defaults
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0002-HBASE-19148-Reevaluate-default-values-of-configurati.patch, 
> HBASE-19148.master.001.patch, HBASE-19148.master.002.patch, 
> HBASE-19148.master.003.patch, HBASE-19148.master.003.patch
>
>
> Remove cruft and mythologies. Make descriptions more digestible. Change 
> defaults given experience.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19148) Reevaluate default values of configurations

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297912#comment-16297912
 ] 

stack commented on HBASE-19148:
---

Go these timeouts:

{code}
[ERROR] Please refer to /testptch/hbase/hbase-server/target/surefire-reports 
for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, 
[date].dumpstream and [date]-jvmRun[N].dumpstream.
[ERROR] ExecutionException The forked VM terminated without properly saying 
goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd /testptch/hbase/hbase-server && 
/usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -enableassertions 
-Dhbase.build.id=2017-12-20T00:56:00Z -Xmx2800m 
-Djava.security.egd=file:/dev/./urandom -Djava.net.preferIPv4Stack=true 
-Djava.awt.headless=true -jar 
/testptch/hbase/hbase-server/target/surefire/surefirebooter7794492890783134322.jar
 /testptch/hbase/hbase-server/target/surefire 2017-12-20T00-56-37_947-jvmRun2 
surefire8203219361689656729tmp surefire_8856831918730071656587tmp
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 1
[ERROR] Crashed tests:
[ERROR] org.apache.hadoop.hbase.TestAcidGuaranteesWithNoInMemCompaction
[ERROR] ExecutionException The forked VM terminated without properly saying 
goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd /testptch/hbase/hbase-server && 
/usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -enableassertions 
-Dhbase.build.id=2017-12-20T00:56:00Z -Xmx2800m 
-Djava.security.egd=file:/dev/./urandom -Djava.net.preferIPv4Stack=true 
-Djava.awt.headless=true -jar 
/testptch/hbase/hbase-server/target/surefire/surefirebooter1002995583245243176.jar
 /testptch/hbase/hbase-server/target/surefire 2017-12-20T00-56-37_947-jvmRun4 
surefire4934381225274999644tmp surefire_8896607335703885400109tmp
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 1
[ERROR] Crashed tests:
[ERROR] org.apache.hadoop.hbase.TestAcidGuaranteesWithAdaptivePolicy
[ERROR] ExecutionException The forked VM terminated without properly saying 
goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd /testptch/hbase/hbase-server && 
/usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -enableassertions 
-Dhbase.build.id=2017-12-20T00:56:00Z -Xmx2800m 
-Djava.security.egd=file:/dev/./urandom -Djava.net.preferIPv4Stack=true 
-Djava.awt.headless=true -jar 
/testptch/hbase/hbase-server/target/surefire/surefirebooter7893607961903870294.jar
 /testptch/hbase/hbase-server/target/surefire 2017-12-20T00-56-37_947-jvmRun1 
surefire928706360139890990tmp surefire_9064920373703367929366tmp
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 1
[ERROR] ExecutionException The forked VM terminated without properly saying 
goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd /testptch/hbase/hbase-server && 
/usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -enableassertions 
-Dhbase.build.id=2017-12-20T00:56:00Z -Xmx2800m 
-Djava.security.egd=file:/dev/./urandom -Djava.net.preferIPv4Stack=true 
-Djava.awt.headless=true -jar 
/testptch/hbase/hbase-server/target/surefire/surefirebooter8109115696455571252.jar
 /testptch/hbase/hbase-server/target/surefire 2017-12-20T00-56-37_947-jvmRun5 
surefire531917999007925890tmp surefire_9073220887283439304467tmp
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 1
[ERROR] Crashed tests:
[ERROR] org.apache.hadoop.hbase.tool.TestCanaryTool
[ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: 
ExecutionException The forked VM terminated without properly saying goodbye. VM 
crash or System.exit called?
[ERROR] Command was /bin/sh -c cd /testptch/hbase/hbase-server && 
/usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -enableassertions 
-Dhbase.build.id=2017-12-20T00:56:00Z -Xmx2800m 
-Djava.security.egd=file:/dev/./urandom -Djava.net.preferIPv4Stack=true 
-Djava.awt.headless=true -jar 
/testptch/hbase/hbase-server/target/surefire/surefirebooter7794492890783134322.jar
 /testptch/hbase/hbase-server/target/surefire 2017-12-20T00-56-37_947-jvmRun2 
surefire8203219361689656729tmp surefire_8856831918730071656587tmp
[ERROR] Error occurred in starting fork, check output in log
{code}

I tried the above locally w/ the patch in place and they seem fine. Let me 
retry.

> Reevaluate default values of configurations
> ---
>
> Key: HBASE-19148
> URL: https://issues.apache.org/jira/browse/HBASE-19148
> Project: HBase
>  Issue Type: Bug
>  Components: defaults
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0002-HBASE-19148-Reevaluate-default-values-of-configurati.patch, 
> 

[jira] [Assigned] (HBASE-19520) Add UTs for the new lock type PEER

[Guanghao Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang reassigned HBASE-19520:
--

Assignee: Guanghao Zhang

> Add UTs for the new lock type PEER
> --
>
> Key: HBASE-19520
> URL: https://issues.apache.org/jira/browse/HBASE-19520
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Guanghao Zhang
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18625) Splitting of region with replica, doesn't update region list in serverHolding. A server crash leads to overlap.

[ramkrishna.s.vasudevan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297907#comment-16297907
 ] 

ramkrishna.s.vasudevan commented on HBASE-18625:


I thought after RS restart is also needed. But since this RegionState is in 
Master I think clearing there is good enough. Sorry for the noise. Patch LGTM. 
+1

> Splitting of region with replica, doesn't update region list in 
> serverHolding. A server crash leads to overlap.
> ---
>
> Key: HBASE-18625
> URL: https://issues.apache.org/jira/browse/HBASE-18625
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.2.6
>Reporter: Igloo
>Assignee: huaxiang sun
> Fix For: 1.2.8
>
> Attachments: HBASE-18625-branch-1-v001.patch, 
> HBASE-18625-branch-1-v002.patch
>
>
> The situation can appear in following steps in release hbase1.2.6
> 1. create 'testtable', 'info', {REGION_REPLICATION=>2}
> 2. write somerecords into 'testtable'
> 3. split the table 'testtable'
> 4. after the spliting, the serverHoldings in RegionStates still holds the 
> regioninfo for the replica of parent region
> 5. restart the regionserver where the parent replica-region located
> 6. the offlined replica of parent region will be assigned in 
> ServerCrashProcedure. 
> hbase hbck 'testtable‘
> ERROR: Region { meta => null, hdfs => null, deployed => 
> qabb-qa-hdp-hbase1,16020,1503022958093;testtable,,1503022907686_0001.42d11cfe195b3cc4d08b2c078a687f6d
> ., replicaId => 1 } not in META, but deployed on 
> qabb-qa-hdp-hbase1,16020,1503022958093
>  18 ERROR: No regioninfo in Meta or HDFS. { meta => null, hdfs => null, 
> deployed => 
> qabb-qa-hdp-hbase1,16020,1503022958093;testtable,,1503022907686_0001.42d11cfe 
>195b3cc4d08b2c078a687f6d., replicaId => 1 }



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19116) Currently the tail of hfiles with CellComparator* classname makes it so hbase1 can't open hbase2 written hfiles; fix

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297906#comment-16297906
 ] 

stack commented on HBASE-19116:
---

Just tried a branch-2 starting over data written by an 0.98. This is the 
exception to fix:

{code}
382603 2017-12-19 21:25:08,333 DEBUG 
[StoreOpener-8e6e2b71d8a413561d9103b0cb6e4686-1] regionserver.HStore: loaded 
hdfs://ve0524.halxg.cloudera.com:8020/hbase/data/default/ycsb/8e6e2b71d8a413561d9103b0cb6e4686/family/3633fe2e03ad4cceb0d2018f1f0d76da,
 isReference=false, isBulkLoadResult=false, seqid=341101   6, 
majorCompaction=false
382604 2017-12-19 21:25:08,339 DEBUG [StoreFileOpenerThread-family-1] 
hfile.HFile: Opening HFile v2 with v3 reader
382605 2017-12-19 21:25:08,342 ERROR [StoreFileOpenerThread-family-1] 
regionserver.StoreFileReader: Error reading bloom filter meta for 
GENERAL_BLOOM_META -- proceeding without
382606 java.io.IOException: java.lang.ClassNotFoundException: 
org.apache.hadoop.hbase.KeyValue$RawBytesComparator
382607   at 
org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.getComparatorClass(FixedFileTrailer.java:583)
382608   at 
org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.createComparator(FixedFileTrailer.java:592)
382609   at 
org.apache.hadoop.hbase.io.hfile.CompoundBloomFilter.(CompoundBloomFilter.java:79)
382610   at 
org.apache.hadoop.hbase.util.BloomFilterFactory.createFromMeta(BloomFilterFactory.java:104)
382611   at 
org.apache.hadoop.hbase.regionserver.StoreFileReader.loadBloomfilter(StoreFileReader.java:480)
382612   at 
org.apache.hadoop.hbase.regionserver.HStoreFile.open(HStoreFile.java:426)
382613   at 
org.apache.hadoop.hbase.regionserver.HStoreFile.initReader(HStoreFile.java:461)
382614   at 
org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:665)
382615   at 
org.apache.hadoop.hbase.regionserver.HStore.lambda$openStoreFiles$0(HStore.java:533)
382616   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
382617   at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
382618   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
382619   at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
382620   at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
382621   at java.lang.Thread.run(Thread.java:745)
382622 Caused by: java.lang.ClassNotFoundException: 
org.apache.hadoop.hbase.KeyValue$RawBytesComparator
382623   at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
382624   at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
382625   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
382626   at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
382627   at java.lang.Class.forName0(Native Method)
382628   at java.lang.Class.forName(Class.java:264)
382629   at 
org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.getComparatorClass(FixedFileTrailer.java:581)
382630   ... 14 more
{code}

> Currently the tail of hfiles with CellComparator* classname makes it so 
> hbase1 can't open hbase2 written hfiles; fix
> 
>
> Key: HBASE-19116
> URL: https://issues.apache.org/jira/browse/HBASE-19116
> Project: HBase
>  Issue Type: Sub-task
>  Components: HFile, migration
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> See tail of HBASE-19052 for discussion which concludes we should try and make 
> it so operators do not have to go to latest hbase version before they 
> upgrade, at least if we can avoid it.
> The necessary change of our default comparator from KV to Cell naming has 
> hfiles with tails that have the classname CellComparator in them in place of 
> KeyValueComparator. If an hbase1 tries to open them, it will fail not having 
> a CellComparator in its classpath (We have name of comparator in tail because 
> different files require different comparators... perhaps we write an alias 
> instead of a class one day... TODO). HBASE-16189 and HBASE-19052 are about 
> trying to carry knowledge of hbase2 back to hbase1, a brittle approach making 
> it so operators will have to upgrade to the latest branch-1 before they can 
> go to hbase2.
> This issue is about undoing our writing of an incompatible (to hbase1) tail, 
> not unless we really have to (and it sounds like we could do without writing 
> an incompatible tail) to see if we can avoid requiring operators go to 
> lastest branch-1 (we may end up needing this but lets a have a really good 
> reason for it if we do).
> Oh, let this filing be an answer to our [~anoop.hbase]'s old high-level 
> question over in HBASE-16189:
> bq. ...means when rolling upgrade done to 2.0, first users have 

[jira] [Commented] (HBASE-19558) TestRegionsOnMasterOptions hack so it works reliablly

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297874#comment-16297874
 ] 

Hudson commented on HBASE-19558:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4257 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/4257/])
HBASE-19558 TestRegionsOnMasterOptions hack so it works reliably (stack: rev 
3c78ad52157576641ee6b42e874b5015b792f34d)
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestRegionsOnMasterOptions.java


> TestRegionsOnMasterOptions hack so it works reliablly
> -
>
> Key: HBASE-19558
> URL: https://issues.apache.org/jira/browse/HBASE-19558
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
>
> See tail of HBASE-19533 for description of problem; our shutdown of a cluster 
> needs work. Meantime hack the test so it doesn't hang on close so it passes 
> reliably again.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19556) Remove TestAssignmentManager#testGoodSplit, which no longer make sense

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297872#comment-16297872
 ] 

Hudson commented on HBASE-19556:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4257 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/4257/])
HBASE-19556: Remove TestAssignmentManager#testGoodSplit, which no longer 
(stack: rev 70608acf289f6de4aeb8f159667a1f21397ec433)
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestAssignmentManager.java


> Remove TestAssignmentManager#testGoodSplit, which no longer make sense
> --
>
> Key: HBASE-19556
> URL: https://issues.apache.org/jira/browse/HBASE-19556
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 2.0.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19556-master-v1.patch
>
>
> {quote}
> [ERROR] 
> testGoodSplit(org.apache.hadoop.hbase.master.assignment.TestAssignmentManager)
>   Time elapsed: 0.478 s  <<< ERROR!
> java.io.IOException: 5a50732f7cb3a05dd3a297bacbc34943 NOT splittable
> {quote}
> GoodSplitExecutor can only mock some functions in RSProcedureDispatcher, 
> however, to test split, we need to create(or mock) a real table on region 
> server side. And GoodSplitExecutor can not mock those kinds of function.
> And, similar test has already been covered in TestSplitTableRegionProcedure, 
> we no longer need to test it again in TestAssignmentManager



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-15482) Provide an option to skip calculating block locations for SnapshotInputFormat

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297873#comment-16297873
 ] 

Hudson commented on HBASE-15482:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4257 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/4257/])
HBASE-15482 Provide an option to skip calculating block locations for (tedyu: 
rev 5e7d16a3ceaeec5057474f9bae2d40d306f6dd8e)
* (edit) 
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatTestBase.java
* (edit) 
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapred/TestTableSnapshotInputFormat.java
* (edit) 
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableSnapshotInputFormat.java
* (edit) 
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java


> Provide an option to skip calculating block locations for SnapshotInputFormat
> -
>
> Key: HBASE-15482
> URL: https://issues.apache.org/jira/browse/HBASE-15482
> Project: HBase
>  Issue Type: Improvement
>  Components: mapreduce
>Reporter: Liyin Tang
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 2.0.0-beta-1
>
> Attachments: 15482.v3.txt, HBASE-15482.master.000.patch, 
> HBASE-15482.master.001.patch, HBASE-15482.master.002.patch, 
> HBASE-15482.master.003.patch
>
>
> When a MR job is reading from SnapshotInputFormat, it needs to calculate the 
> splits based on the block locations in order to get best locality. However, 
> this process may take a long time for large snapshots. 
> In some setup, the computing layer, Spark, Hive or Presto could run out side 
> of HBase cluster. In these scenarios, the block locality doesn't matter. 
> Therefore, it will be great to have an option to skip calculating the block 
> locations for every job. That will super useful for the Hive/Presto/Spark 
> connectors.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19494) Create simple WALKey filter that can be plugged in on the Replication Sink

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297871#comment-16297871
 ] 

Hudson commented on HBASE-19494:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4257 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/4257/])
HBASE-19494 Create simple WALKey filter that can be plugged in on the (stack: 
rev 32f6fd41c274a955338b7c43ab80309b9adbba0d)
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java
* (add) 
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestWALEntrySinkFilter.java
* (add) 
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/WALEntrySinkFilter.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/WALEntryFilter.java


> Create simple WALKey filter that can be plugged in on the Replication Sink
> --
>
> Key: HBASE-19494
> URL: https://issues.apache.org/jira/browse/HBASE-19494
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0003-HBASE-19494-Create-simple-WALKey-filter-that-can-be-.patch, 
> HBASE-19494.master.001.patch
>
>
> hbase-indexer used to look at WALKeys on the sink to see if their time of 
> creation was before the time at which the replication stream was enabled.
> In the parent redo, there is no means for doing this anymore (because WALKey 
> used to be Private and because to get at the WALKey in the Sink, you had to 
> override all of the Replication which meant importing a million Private 
> objects...).
> This issue is about adding a simple filter to Replication on the sink-side 
> that just takes a WALKey (now InterfaceAudience LimitedPrivate and recently 
> made read-only).
> Assigned myself. Need to do this so hbase-indexer can move to hbase2.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19561) maxCacheSize in CacheEvictionStats can't be accumulated repeatedly When dealing with each region

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297865#comment-16297865
 ] 

Ted Yu commented on HBASE-19561:


lgtm

> maxCacheSize in CacheEvictionStats can't be accumulated repeatedly When 
> dealing with each region
> 
>
> Key: HBASE-19561
> URL: https://issues.apache.org/jira/browse/HBASE-19561
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
> Attachments: HBASE-19561.master.001.patch
>
>
> {code:title=HRegionServer.java|borderStyle=solid}
>   public CacheEvictionStats clearRegionBlockCache(Region region) {
> BlockCache blockCache = this.getCacheConfig().getBlockCache();
> long evictedBlocks = 0;
> for(Store store : region.getStores()) {
>   for(StoreFile hFile : store.getStorefiles()) {
> evictedBlocks += 
> blockCache.evictBlocksByHfileName(hFile.getPath().getName());
>   }
> }
> return CacheEvictionStats.builder()
> .withEvictedBlocks(evictedBlocks)
> .withMaxCacheSize(blockCache.getMaxSize())
> .build();
>   }
> {code}
> BlockCache is the regionserver level object.If we accumulate this value for 
> each region, we will get a wrong result.
> The value of maxCacheSize should be set to BlockCache.getMaxSize() after 
> processing all the regions on this regionserver.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19561) maxCacheSize in CacheEvictionStats can't be accumulated repeatedly When dealing with each region

[Zach York (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297864#comment-16297864
 ] 

Zach York commented on HBASE-19561:
---

[~andrewcheng] Good catch! LGTM, +1

> maxCacheSize in CacheEvictionStats can't be accumulated repeatedly When 
> dealing with each region
> 
>
> Key: HBASE-19561
> URL: https://issues.apache.org/jira/browse/HBASE-19561
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
> Attachments: HBASE-19561.master.001.patch
>
>
> {code:title=HRegionServer.java|borderStyle=solid}
>   public CacheEvictionStats clearRegionBlockCache(Region region) {
> BlockCache blockCache = this.getCacheConfig().getBlockCache();
> long evictedBlocks = 0;
> for(Store store : region.getStores()) {
>   for(StoreFile hFile : store.getStorefiles()) {
> evictedBlocks += 
> blockCache.evictBlocksByHfileName(hFile.getPath().getName());
>   }
> }
> return CacheEvictionStats.builder()
> .withEvictedBlocks(evictedBlocks)
> .withMaxCacheSize(blockCache.getMaxSize())
> .build();
>   }
> {code}
> BlockCache is the regionserver level object.If we accumulate this value for 
> each region, we will get a wrong result.
> The value of maxCacheSize should be set to BlockCache.getMaxSize() after 
> processing all the regions on this regionserver.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19561) maxCacheSize in CacheEvictionStats can't be accumulated repeatedly When dealing with each region

[Guangxu Cheng (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guangxu Cheng updated HBASE-19561:
--
Attachment: HBASE-19561.master.001.patch

[~yuzhih...@gmail.com] [~zyork] Can you take a look?Thanks

> maxCacheSize in CacheEvictionStats can't be accumulated repeatedly When 
> dealing with each region
> 
>
> Key: HBASE-19561
> URL: https://issues.apache.org/jira/browse/HBASE-19561
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
> Attachments: HBASE-19561.master.001.patch
>
>
> {code:title=HRegionServer.java|borderStyle=solid}
>   public CacheEvictionStats clearRegionBlockCache(Region region) {
> BlockCache blockCache = this.getCacheConfig().getBlockCache();
> long evictedBlocks = 0;
> for(Store store : region.getStores()) {
>   for(StoreFile hFile : store.getStorefiles()) {
> evictedBlocks += 
> blockCache.evictBlocksByHfileName(hFile.getPath().getName());
>   }
> }
> return CacheEvictionStats.builder()
> .withEvictedBlocks(evictedBlocks)
> .withMaxCacheSize(blockCache.getMaxSize())
> .build();
>   }
> {code}
> BlockCache is the regionserver level object.If we accumulate this value for 
> each region, we will get a wrong result.
> The value of maxCacheSize should be set to BlockCache.getMaxSize() after 
> processing all the regions on this regionserver.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19561) maxCacheSize in CacheEvictionStats can't be accumulated repeatedly When dealing with each region

[Guangxu Cheng (JIRA)]

Guangxu Cheng created HBASE-19561:
-

 Summary: maxCacheSize in CacheEvictionStats can't be accumulated 
repeatedly When dealing with each region
 Key: HBASE-19561
 URL: https://issues.apache.org/jira/browse/HBASE-19561
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Guangxu Cheng
Assignee: Guangxu Cheng


{code:title=HRegionServer.java|borderStyle=solid}
  public CacheEvictionStats clearRegionBlockCache(Region region) {
BlockCache blockCache = this.getCacheConfig().getBlockCache();
long evictedBlocks = 0;

for(Store store : region.getStores()) {
  for(StoreFile hFile : store.getStorefiles()) {
evictedBlocks += 
blockCache.evictBlocksByHfileName(hFile.getPath().getName());
  }
}

return CacheEvictionStats.builder()
.withEvictedBlocks(evictedBlocks)
.withMaxCacheSize(blockCache.getMaxSize())
.build();
  }
{code}

BlockCache is the regionserver level object.If we accumulate this value for 
each region, we will get a wrong result.
The value of maxCacheSize should be set to BlockCache.getMaxSize() after 
processing all the regions on this regionserver.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19547) HBase fails building on AArch64 due to asciidoctor-maven-plugin.

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297854#comment-16297854
 ] 

Ted Yu commented on HBASE-19547:


Several occurrences of the following in output:
{code}
[ERROR] Command was /bin/sh -c cd /testptch/hbase/hbase-server && 
/usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -enableassertions 
-Dhbase.build.id=2017-12-19T04:24:11Z -Xmx2800m 
-Djava.security.egd=file:/dev/./urandom -Djava.net.preferIPv4Stack=true 
-Djava.awt.headless=true -jar 
/testptch/hbase/hbase-server/target/surefire/surefirebooter8863999073719792036.jar
 /testptch/hbase/hbase-server/target/surefire 2017-12-19T04-24-17_238-jvmRun3 
surefire1877980246042989754tmp surefire_11936792857066479530434tmp
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 1
[ERROR] Crashed tests:
{code}
Please run test suite again.

> HBase fails building on AArch64 due to asciidoctor-maven-plugin.
> 
>
> Key: HBASE-19547
> URL: https://issues.apache.org/jira/browse/HBASE-19547
> Project: HBase
>  Issue Type: Bug
>  Components: build, hbase, jruby
>Affects Versions: 3.0.0
>Reporter: Yuqi Gu
>Assignee: Yuqi Gu
> Attachments: HBASE-19547.patch
>
>
> HBase fails building on AArch64 due to asciidoctor-maven-plugin.
> {code:java}
> mvn clean site package install -DskipTests 
> [ERROR] Failed to execute 
> goal org.asciidoctor:asciidoctor-maven-plugin:1.5.5:
> process-asciidoc (output-pdf) on project hbase: 
> Execution output-pdf of goal 
> org.asciidoctor:asciidoctor-maven-plugin:1.5.5:
> process-asciidoc failed: (NotImplementedError)
>  fstat unimplemented unsupported or native support failed to load ->
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19560) create make-rc.sh for hbase-thirdparty

[Mike Drob (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob updated HBASE-19560:
--
Fix Version/s: thirdparty-2.0.1

> create make-rc.sh for hbase-thirdparty
> --
>
> Key: HBASE-19560
> URL: https://issues.apache.org/jira/browse/HBASE-19560
> Project: HBase
>  Issue Type: Task
>  Components: hbase-thirdparty
>Reporter: Mike Drob
> Fix For: thirdparty-2.0.1
>
>
> Make it easier to automate the process...
> Some notes, may not correspond to exact commands I ran...
> {noformat}
> mvn clean deploy -Papache-release
> git tag -s HEAD
> git archive HEAD -o release.tar.gz
> gpg --sign release.tar.gz
> gpg --print-mds
> gpg --print-md MD5
> gpg --print-md SHA512
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19560) create make-rc.sh for hbase-thirdparty

[Mike Drob (JIRA)]

Mike Drob created HBASE-19560:
-

 Summary: create make-rc.sh for hbase-thirdparty
 Key: HBASE-19560
 URL: https://issues.apache.org/jira/browse/HBASE-19560
 Project: HBase
  Issue Type: Task
  Components: hbase-thirdparty
Reporter: Mike Drob


Make it easier to automate the process...

Some notes, may not correspond to exact commands I ran...

{noformat}
mvn clean deploy -Papache-release
git tag -s HEAD
git archive HEAD -o release.tar.gz
gpg --sign release.tar.gz
gpg --print-mds
gpg --print-md MD5
gpg --print-md SHA512
{noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19557) Build and release source jars for hbase-shaded-client and others

[Jerry He (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jerry He updated HBASE-19557:
-
Issue Type: Sub-task  (was: Improvement)
Parent: HBASE-18800

> Build and release source jars for hbase-shaded-client and others
> 
>
> Key: HBASE-19557
> URL: https://issues.apache.org/jira/browse/HBASE-19557
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 1.3.1, 1.2.6
>Reporter: Jerry He
>
> It seems that currently we don't build and release source jars for 
> hbase-shaded-client (and server or mapreduce).  IDEs will complain from the 
> dependent users. We should provide them.
> http://central.maven.org/maven2/org/apache/hbase/hbase-shaded-client/1.3.1/



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (HBASE-15482) Provide an option to skip calculating block locations for SnapshotInputFormat

[Xiang Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297060#comment-16297060
 ] 

Xiang Li edited comment on HBASE-15482 at 12/20/17 2:56 AM:


Hi [~tedyu], [~jerryhe], thanks for your comments and guide! 
Patch 003 is uploaded to address the following changes mainly:
* Simplify the logic in light of 15482.v3.txt. Besides, add the logic to
** Check if numTopsAtMost < 1 (which is invalid)
** Check if top is 1. When it is 1, return top host directly.
* Adjust testGetBestLocations()
* Change the conf key string from 
{{hbase.TableSnapshotInputFormat.locality.enable}} into 
{{hbase.TableSnapshotInputFormat.locality.enabled}}, by using "enabled" instead 
of "enable", as I see most of the conf key strings are using "enabled"


was (Author: water):
Hi [~tedyu], [~jerryhe], thanks for your comments and guide! 
Patch 003 is uploaded to address the following changes mainly:
* Simply the logic in light of 15482.v3.txt. Besides, add the logic to
** Check if numTopsAtMost < 1 (which is invalid)
** Check if top is 1. When it is 1, return top host directly.
* Adjust testGetBestLocations()
* Change the conf key string from 
{{hbase.TableSnapshotInputFormat.locality.enable}} into 
{{hbase.TableSnapshotInputFormat.locality.enabled}}, by using "enabled" instead 
of "enable", as I see most of the conf key strings are using "enabled"

> Provide an option to skip calculating block locations for SnapshotInputFormat
> -
>
> Key: HBASE-15482
> URL: https://issues.apache.org/jira/browse/HBASE-15482
> Project: HBase
>  Issue Type: Improvement
>  Components: mapreduce
>Reporter: Liyin Tang
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 2.0.0-beta-1
>
> Attachments: 15482.v3.txt, HBASE-15482.master.000.patch, 
> HBASE-15482.master.001.patch, HBASE-15482.master.002.patch, 
> HBASE-15482.master.003.patch
>
>
> When a MR job is reading from SnapshotInputFormat, it needs to calculate the 
> splits based on the block locations in order to get best locality. However, 
> this process may take a long time for large snapshots. 
> In some setup, the computing layer, Spark, Hive or Presto could run out side 
> of HBase cluster. In these scenarios, the block locality doesn't matter. 
> Therefore, it will be great to have an option to skip calculating the block 
> locations for every job. That will super useful for the Hive/Presto/Spark 
> connectors.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (HBASE-15482) Provide an option to skip calculating block locations for SnapshotInputFormat

[Xiang Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297084#comment-16297084
 ] 

Xiang Li edited comment on HBASE-15482 at 12/20/17 2:56 AM:


Regarding the UT:
* For mapred, the number of splits generated(=10) is exactly the same as 
numRegions specified(=10), while only 8 of them have locations not being an 
empty array.
* For mapreduce, only 8 splits are generated when there are 10 regions.


was (Author: water):
Regarding the UT:
* For mapred, the number of splits generated(=10) is exactly the same as 
numRegions specified(=10), while only 8 of them have locations not being an 
empty array.
* For mapreduce, only 8 splits are generated when there is 10 regions.

> Provide an option to skip calculating block locations for SnapshotInputFormat
> -
>
> Key: HBASE-15482
> URL: https://issues.apache.org/jira/browse/HBASE-15482
> Project: HBase
>  Issue Type: Improvement
>  Components: mapreduce
>Reporter: Liyin Tang
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 2.0.0-beta-1
>
> Attachments: 15482.v3.txt, HBASE-15482.master.000.patch, 
> HBASE-15482.master.001.patch, HBASE-15482.master.002.patch, 
> HBASE-15482.master.003.patch
>
>
> When a MR job is reading from SnapshotInputFormat, it needs to calculate the 
> splits based on the block locations in order to get best locality. However, 
> this process may take a long time for large snapshots. 
> In some setup, the computing layer, Spark, Hive or Presto could run out side 
> of HBase cluster. In these scenarios, the block locality doesn't matter. 
> Therefore, it will be great to have an option to skip calculating the block 
> locations for every job. That will super useful for the Hive/Presto/Spark 
> connectors.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19218) Master stuck thinking hbase:namespace is assigned after restart preventing intialization

[Mike Drob (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297826#comment-16297826
 ] 

Mike Drob commented on HBASE-19218:
---

I was chatting with our [~esteban] about this earlier and it sounded like he's 
seen this outside of RSGroups, possibly related to regions on master?

> Master stuck thinking hbase:namespace is assigned after restart preventing 
> intialization
> 
>
> Key: HBASE-19218
> URL: https://issues.apache.org/jira/browse/HBASE-19218
> Project: HBase
>  Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Umesh Agashe
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> hbase-hbase-master-ctr-e134-1499953498516-282290-01-03.hwx.site.log.zip, 
> hbase-site.xml
>
>
> Our [~romil.choksi] brought this one to my attention after trying to get some 
> cluster tests running.
> The Master seems to have gotten stuck never initializing after it thinks that 
> hbase:namespace was already deployed on the cluster when it actually was not. 
> On a Master restart, it reads the location out of meta and assumes that it's 
> there (I assume this invalid entry is the issue):
> {noformat}
> 2017-11-08 00:29:17,556 INFO  
> [ctr-e134-1499953498516-282290-01-03:2.masterManager] 
> assignment.RegionStateStore: Load hbase:meta entry region={ENCODED => 
> f147f204a579b885c351bdc0a7ebbf94, NAME => 
> 'hbase:namespace,,1510084256045.f147f204a579b885c351bdc0a7ebbf94.', STARTKEY 
> => '', ENDKEY => ''} regionState=OPENING 
> lastHost=ctr-e134-1499953498516-282290-01-05.hwx.site,16020,1510084579728 
> regionLocation=ctr-e134-1499953498516-282290-01-05.hwx.site,16020,1510100695534
> {noformat}
> Prior to this, the RS5 went through the ServerCrashProcedure, but it looks 
> like this bailed out unexpectedly:
> {noformat}
> 2017-11-08 00:25:25,187 WARN  
> [ctr-e134-1499953498516-282290-01-03:2.masterManager] 
> master.ServerManager: Expiration of 
> ctr-e134-1499953498516-282290-01-05.hwx.site,16020,1510084579728 but 
> server not online
> 2017-11-08 00:25:25,187 INFO  [ProcExecWrkr-5] 
> procedure.ServerCrashProcedure: Start pid=36, 
> state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure 
> server=ctr-e134-1499953498516-282290-01-03.hwx.site,16020,1510084580111, 
> splitWal=t
> rue, meta=false
> 2017-11-08 00:25:25,188 INFO  
> [ctr-e134-1499953498516-282290-01-03:2.masterManager] 
> master.ServerManager: Processing expiration of 
> ctr-e134-1499953498516-282290-01-05.hwx.site,16020,1510084579728 on 
> ctr-e134-1499953498516-28
> 2290-01-03.hwx.site,2,1510100690324
> ...
> 2017-11-08 00:25:27,211 ERROR [ProcExecWrkr-22] procedure2.ProcedureExecutor: 
> CODE-BUG: Uncaught runtime exception: pid=40, ppid=37, 
> state=RUNNABLE:REGION_TRANSITION_QUEUE; AssignProcedure 
> table=hbase:namespace, region=f147f204a579b885c351bdc0a7ebbf94
> java.lang.NullPointerException
> at 
> java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
> at 
> org.apache.hadoop.hbase.procedure2.RemoteProcedureDispatcher.addOperationToNode(RemoteProcedureDispatcher.java:171)
> at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.addToRemoteDispatcher(RegionTransitionProcedure.java:223)
> at 
> org.apache.hadoop.hbase.master.assignment.AssignProcedure.updateTransition(AssignProcedure.java:252)
> at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:309)
> at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:82)
> at 
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1452)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1221)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:77)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1731)
> 2017-11-08 00:25:27,239 FATAL [ProcExecWrkr-22] procedure2.ProcedureExecutor: 
> CODE-BUG: Uncaught runtime exception for pid=37, 
> state=FAILED:SERVER_CRASH_FINISH, exception=java.lang.NullPointerException 
> via CODE-BUG: Uncaught runtime exception: pid=40, ppid=37, 
> state=RUNNABLE:REGION_TRANSITION_QUEUE; AssignProcedure 
> table=hbase:namespace, 
> region=f147f204a579b885c351bdc0a7ebbf94:java.lang.NullPointerException; 
> ServerCrashProcedure 
> server=ctr-e134-1499953498516-282290-01-05.hwx.site,16020,1510084579728, 
> splitWal=true, meta=false

[jira] [Commented] (HBASE-19148) Reevaluate default values of configurations

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297825#comment-16297825
 ] 

Hadoop QA commented on HBASE-19148:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
 5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  6m 
19s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
45s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
10s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  1m  
9s{color} | {color:red} hbase-server: The patch generated 1 new + 262 unchanged 
- 0 fixed = 263 total (was 262) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
45s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
20m  5s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
44s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
11s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}118m 53s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
35s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}164m 25s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19148 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12902950/HBASE-19148.master.003.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  xml  findbugs  shadedjars 
 hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 9c9330bfb1a3 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11 
12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 

[jira] [Comment Edited] (HBASE-15482) Provide an option to skip calculating block locations for SnapshotInputFormat

[Xiang Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297060#comment-16297060
 ] 

Xiang Li edited comment on HBASE-15482 at 12/20/17 2:50 AM:


Hi [~tedyu], [~jerryhe], thanks for your comments and guide! 
Patch 003 is uploaded to address the following changes mainly:
* Simply the logic in light of 15482.v3.txt. Besides, add the logic to
** Check if numTopsAtMost < 1 (which is invalid)
** Check if top is 1. When it is 1, return top host directly.
* Adjust testGetBestLocations()
* Change the conf key string from 
{{hbase.TableSnapshotInputFormat.locality.enable}} into 
{{hbase.TableSnapshotInputFormat.locality.enabled}}, by using "enabled" instead 
of "enable", as I see most of the conf key strings are using "enabled"


was (Author: water):
Hi [~tedyu], [~jerryhe], thanks for your comments and guide! 
Patch 003 is uploaded to address the following changes mainly:
* Simple the logic in light of 15482.v3.txt. Besides, add the logic to
** Check if numTopsAtMost < 1 (which is invalid)
** Check if top is 1. When it is 1, return top host directly.
* Adjust testGetBestLocations()
* Change the conf key string from 
{{hbase.TableSnapshotInputFormat.locality.enable}} into 
{{hbase.TableSnapshotInputFormat.locality.enabled}}, by using "enabled" instead 
of "enable", as I see most of the conf key strings are using "enabled"

> Provide an option to skip calculating block locations for SnapshotInputFormat
> -
>
> Key: HBASE-15482
> URL: https://issues.apache.org/jira/browse/HBASE-15482
> Project: HBase
>  Issue Type: Improvement
>  Components: mapreduce
>Reporter: Liyin Tang
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 2.0.0-beta-1
>
> Attachments: 15482.v3.txt, HBASE-15482.master.000.patch, 
> HBASE-15482.master.001.patch, HBASE-15482.master.002.patch, 
> HBASE-15482.master.003.patch
>
>
> When a MR job is reading from SnapshotInputFormat, it needs to calculate the 
> splits based on the block locations in order to get best locality. However, 
> this process may take a long time for large snapshots. 
> In some setup, the computing layer, Spark, Hive or Presto could run out side 
> of HBase cluster. In these scenarios, the block locality doesn't matter. 
> Therefore, it will be great to have an option to skip calculating the block 
> locations for every job. That will super useful for the Hive/Presto/Spark 
> connectors.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18970) The version of jruby we use now can't get interactive input from prompt

[Mike Drob (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob updated HBASE-18970:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Pushed, thanks for the patch, [~avirmani]!

> The version of jruby we use now can't get interactive input from prompt
> ---
>
> Key: HBASE-18970
> URL: https://issues.apache.org/jira/browse/HBASE-18970
> Project: HBase
>  Issue Type: Task
>  Components: shell
>Reporter: Chia-Ping Tsai
>Assignee: AMIT VIRMANI
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: HBase-18970.branch-2.001.patch, samplecmd.txt, 
> test_shell-Output.txt, test_shell.sh
>
>
> case 1: press enter
> {code}
> hbase(main):002:0> disable_all 'chia(.*)'
> chia_1
>   
>
> Disable the above 1 tables (y/n)?
> y^M
> {code}
> case 2: press ctrl-j
> {code}
> hbase(main):001:0> disable_all 'chia(.*)'
> chia_1
>   
>
> Disable the above 1 tables (y/n)?
> y^J1 tables successfully disabled
> Took 5.0059 seconds  
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19559) Fix TestLogRolling.testLogRollOnDatanodeDeath

[Duo Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-19559:
--
Component/s: wal
 test

> Fix TestLogRolling.testLogRollOnDatanodeDeath
> -
>
> Key: HBASE-19559
> URL: https://issues.apache.org/jira/browse/HBASE-19559
> Project: HBase
>  Issue Type: Bug
>  Components: test, wal
>Reporter: Duo Zhang
>
> It appears several times on the failing list of pre commit result.
> The test code is a bit confusing to me. Need to dig more.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19483) Add proper privilege check for rsgroup commands

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297805#comment-16297805
 ] 

Ted Yu commented on HBASE-19483:


[~appy]:
Can you take a look ?

> Add proper privilege check for rsgroup commands
> ---
>
> Key: HBASE-19483
> URL: https://issues.apache.org/jira/browse/HBASE-19483
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Guangxu Cheng
> Fix For: 1.4.1, 1.5.0, 2.0.0-beta-2
>
> Attachments: HBASE-19483.master.001.patch, 
> HBASE-19483.master.002.patch, HBASE-19483.master.003.patch, 
> HBASE-19483.master.004.patch, HBASE-19483.master.005.patch
>
>
> Currently list_rsgroups command can be executed by any user.
> This is inconsistent with other list commands such as list_peers and 
> list_peer_configs.
> We should add proper privilege check for list_rsgroups command.
> privilege check should be added for get_table_rsgroup / get_server_rsgroup / 
> get_rsgroup commands.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19559) Fix TestLogRolling.testLogRollOnDatanodeDeath

[Duo Zhang (JIRA)]

Duo Zhang created HBASE-19559:
-

 Summary: Fix TestLogRolling.testLogRollOnDatanodeDeath
 Key: HBASE-19559
 URL: https://issues.apache.org/jira/browse/HBASE-19559
 Project: HBase
  Issue Type: Bug
Reporter: Duo Zhang


It appears several times on the failing list of pre commit result.

The test code is a bit confusing to me. Need to dig more.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17513) Thrift Server 1 uses different QOP settings than RPC and Thrift Server 2 and can easily be misconfigured so there is no encryption when the operator expects it.

[Reid Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297803#comment-16297803
 ] 

Reid Chan commented on HBASE-17513:
---

Thank you [~tedyu].
Hi [~busbey], would you mind taking some time to have a look.

> Thrift Server 1 uses different QOP settings than RPC and Thrift Server 2 and 
> can easily be misconfigured so there is no encryption when the operator 
> expects it.
> 
>
> Key: HBASE-17513
> URL: https://issues.apache.org/jira/browse/HBASE-17513
> Project: HBase
>  Issue Type: Bug
>  Components: documentation, security, Thrift, Usability
>Affects Versions: 2.0.0, 1.2.0, 1.3.0, 0.98.15, 1.0.3, 1.1.3
>Reporter: Sean Busbey
>Assignee: Reid Chan
>Priority: Critical
> Fix For: 2.0.0, 1.3.2, 1.4.1, 1.2.8
>
> Attachments: HBASE-17513.branch-1.001.patch, 
> HBASE-17513.master.001.patch, HBASE-17513.master.002.patch
>
>
> As of HBASE-14400 the setting {{hbase.thrift.security.qop}} was unified to 
> behave the same as the general HBase RPC protection. However, this only 
> happened for the Thrift2 server. The Thrift server found in the thrift 
> package (aka Thrift Server 1) still hard codes the old configs of 'auth', 
> 'auth-int', and 'auth-conf'.
> Additionally, these Quality of Protection (qop) settings are used only by the 
> SASL transport. If a user configures the HBase Thrift Server to make use of 
> the HTTP transport (to enable doAs proxying e.g. for Hue) then a QOP setting 
> of 'privacy' or 'auth-conf' won't get them encryption as expected.
> We should
> 1) update {{hbase-thrift/src/main/.../thrift/ThriftServerRunner}} to rely on 
> {{SaslUtil}} to use the same 'authentication', 'integrity', 'privacy' configs 
> in a backward compatible way
> 2) also have ThriftServerRunner warn when both {{hbase.thrift.security.qop}} 
> and {{hbase.regionserver.thrift.http}} are set, since the latter will cause 
> the former to be ignored. (users should be directed to 
> {{hbase.thrift.ssl.enabled}} and related configs to ensure their transport is 
> encrypted when using the HTTP transport.)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18440) ITs and Actions modify immutable TableDescriptors

[Guanghao Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297799#comment-16297799
 ] 

Guanghao Zhang commented on HBASE-18440:


bq. Do you have a JIRA filed for the failed verification? Can you post error 
there?
Thanks for the remind. Let me try it again on master branch. Will file a jira 
after get the  failed verification again.

> ITs and Actions modify immutable TableDescriptors
> -
>
> Key: HBASE-18440
> URL: https://issues.apache.org/jira/browse/HBASE-18440
> Project: HBase
>  Issue Type: Sub-task
>  Components: integration tests
>Reporter: Mike Drob
>Assignee: Guanghao Zhang
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-18440.master.004.patch, 
> HBASE-18440.master.005.patch, HBASE-18440.patch, HBASE-18440.v2.patch, 
> HBASE-18440.v3.patch, HBASE-18440.v3.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19483) Add proper privilege check for rsgroup commands

[Guangxu Cheng (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guangxu Cheng updated HBASE-19483:
--
Attachment: HBASE-19483.master.005.patch

Update patch as [~yuzhih...@gmail.com] suggestions and fix checkstyle warnings. 
Thanks

> Add proper privilege check for rsgroup commands
> ---
>
> Key: HBASE-19483
> URL: https://issues.apache.org/jira/browse/HBASE-19483
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Guangxu Cheng
> Fix For: 1.4.1, 1.5.0, 2.0.0-beta-2
>
> Attachments: HBASE-19483.master.001.patch, 
> HBASE-19483.master.002.patch, HBASE-19483.master.003.patch, 
> HBASE-19483.master.004.patch, HBASE-19483.master.005.patch
>
>
> Currently list_rsgroups command can be executed by any user.
> This is inconsistent with other list commands such as list_peers and 
> list_peer_configs.
> We should add proper privilege check for list_rsgroups command.
> privilege check should be added for get_table_rsgroup / get_server_rsgroup / 
> get_rsgroup commands.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17436) Add facility to provide more information for Other Regions seen on Master UI

[Appy (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297791#comment-16297791
 ] 

Appy commented on HBASE-17436:
--

Looking at 17436.v11.txt. Is that the right patch? (It's the most recent one.) 
I ask because i don't see regionstate.jsp which is there in earlier patch.
Otherwise looks good.



> Add facility to provide more information for Other Regions seen on Master UI
> 
>
> Key: HBASE-17436
> URL: https://issues.apache.org/jira/browse/HBASE-17436
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Janos Gub
>  Labels: ui, usability
> Fix For: 2.0.0
>
> Attachments: 17436.v10.txt, 17436.v11.txt, 17436.v9.txt, 
> HBASE-17436-v2.patch, HBASE-17436-v5.patch, HBASE-17436-v6.patch, 
> HBASE-17436-v6.patch, HBASE-17436-v7.patch, HBASE-17436-v8.patch, 
> HBASE-17436.patch, HBASE-17779-v3.patch, HBASE-17779-v4.patch, Screen Shot 
> 2017-04-24 at 8.43.31 PM.png, Screen Shot 2017-04-26 at 4.39.49 PM.png, 
> Screen Shot 2017-11-08 at 10.29.49 AM.png, Screen Shot 2017-11-08 at 10.30.09 
> AM.png, Screen Shot 2017-11-22 at 4.17.35 PM.png, initial.patch, 
> regionstates.png, table-selected.png, tableregions.png
>
>
> [~rpednekar] and I were looking at a case where the count displayed under 
> Other Regions was high (~1200).
> Since the table page just maintains a count instead of List of region names, 
> it is very difficult for user to determine which regions belong to this 
> category.
> We should provide facility to provide more details for this category (LOG, 
> JMX, etc).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18440) ITs and Actions modify immutable TableDescriptors

[Mike Drob (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297780#comment-16297780
 ] 

Mike Drob commented on HBASE-18440:
---

Thanks for following up on this [~zghaobac]!

Do you have a JIRA filed for the failed verification? Can you post error there?

> ITs and Actions modify immutable TableDescriptors
> -
>
> Key: HBASE-18440
> URL: https://issues.apache.org/jira/browse/HBASE-18440
> Project: HBase
>  Issue Type: Sub-task
>  Components: integration tests
>Reporter: Mike Drob
>Assignee: Guanghao Zhang
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-18440.master.004.patch, 
> HBASE-18440.master.005.patch, HBASE-18440.patch, HBASE-18440.v2.patch, 
> HBASE-18440.v3.patch, HBASE-18440.v3.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-18429) ITs attempt to modify immutable table/column descriptors

[Guanghao Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang resolved HBASE-18429.

Resolution: Fixed
  Assignee: Mike Drob

Resolve this as all sub-tasks done.

> ITs attempt to modify immutable table/column descriptors
> 
>
> Key: HBASE-18429
> URL: https://issues.apache.org/jira/browse/HBASE-18429
> Project: HBase
>  Issue Type: Umbrella
>  Components: integration tests
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
>
> ITs:
> * IntegrationTestIngestWithMOB (HBASE-18419)
> * IntegrationTestDDLMasterFailover (HBASE-18428)
> * IntegrationTestIngestWithEncryption::setUp (HBASE-18440)
> * IntegrationTestBulkLoad::installSlowingCoproc (HBASE-18440)
> Other Related:
> * ChangeBloomFilterAction (HBASE-18419)
> * ChangeCompressionAction (HBASE-18419)
> * ChangeEncodingAction (HBASE-18419)
> * ChangeVersionsAction (HBASE-18419)
> * RemoveColumnAction (HBASE-18419)
> * AddColumnAction::perform (HBASE-18440)
> * ChangeSplitPolicyAction::perform (HBASE-18440)
> * DecreaseMaxHFileSizeAction::perform (HBASE-18440)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18429) ITs attempt to modify immutable table/column descriptors

[Guanghao Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-18429:
---
Fix Version/s: 2.0.0-beta-1

> ITs attempt to modify immutable table/column descriptors
> 
>
> Key: HBASE-18429
> URL: https://issues.apache.org/jira/browse/HBASE-18429
> Project: HBase
>  Issue Type: Umbrella
>  Components: integration tests
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
>
> ITs:
> * IntegrationTestIngestWithMOB (HBASE-18419)
> * IntegrationTestDDLMasterFailover (HBASE-18428)
> * IntegrationTestIngestWithEncryption::setUp (HBASE-18440)
> * IntegrationTestBulkLoad::installSlowingCoproc (HBASE-18440)
> Other Related:
> * ChangeBloomFilterAction (HBASE-18419)
> * ChangeCompressionAction (HBASE-18419)
> * ChangeEncodingAction (HBASE-18419)
> * ChangeVersionsAction (HBASE-18419)
> * RemoveColumnAction (HBASE-18419)
> * AddColumnAction::perform (HBASE-18440)
> * ChangeSplitPolicyAction::perform (HBASE-18440)
> * DecreaseMaxHFileSizeAction::perform (HBASE-18440)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-10092) Move to slf4j

[Appy (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297772#comment-16297772
 ] 

Appy commented on HBASE-10092:
--

Facepalm. I had a review on RB which i forgot to publish yesterday. Ignore it 
now.

> Move to slf4j
> -
>
> Key: HBASE-10092
> URL: https://issues.apache.org/jira/browse/HBASE-10092
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Assignee: Balazs Meszaros
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 10092.txt, 10092v2.txt, HBASE-10092-preview-v0.patch, 
> HBASE-10092.master.001.patch, HBASE-10092.master.002.patch, 
> HBASE-10092.master.003.patch, HBASE-10092.patch
>
>
> Allows logging with less friction.  See http://logging.apache.org/log4j/2.x/  
> This rather radical transition can be done w/ minor change given they have an 
> adapter for apache's logging, the one we use.  They also have and adapter for 
> slf4j so we likely can remove at least some of the 4 versions of this module 
> our dependencies make use of.
> I made a start in attached patch but am currently stuck in maven dependency 
> resolve hell courtesy of our slf4j.  Fixing will take some concentration and 
> a good net connection, an item I currently lack.  Other TODOs are that will 
> need to fix our little log level setting jsp page -- will likely have to undo 
> our use of hadoop's tool here -- and the config system changes a little.
> I will return to this project soon.  Will bring numbers.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19525) RS side changes for moving peer modification from zk watcher to procedure

[Zheng Hu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297771#comment-16297771
 ] 

Zheng Hu commented on HBASE-19525:
--

[~Apache9],  Let me rebase & fix the failed UTs, Will put the patch.v2  to RB.  
Thanks . 

> RS side changes for moving peer modification from zk watcher to procedure
> -
>
> Key: HBASE-19525
> URL: https://issues.apache.org/jira/browse/HBASE-19525
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2, Replication
>Reporter: Duo Zhang
>Assignee: Zheng Hu
> Fix For: HBASE-19397
>
> Attachments: HBASE-19525.HBASE-19397.v1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19468) FNFE during scans and flushes

[Thiruvel Thirumoolan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297769#comment-16297769
 ] 

Thiruvel Thirumoolan commented on HBASE-19468:
--

[~ram_krish] +1

I also verified with a unit test that files are cleaned up even when scanner 
expires.

I will run ITBLL with this patch sometime in next few days. I guess this patch 
can get in and not have to wait for that.

Thanks Ram!

> FNFE during scans and flushes
> -
>
> Key: HBASE-19468
> URL: https://issues.apache.org/jira/browse/HBASE-19468
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver, Scanners
>Affects Versions: 1.3.1
>Reporter: Thiruvel Thirumoolan
>Priority: Critical
> Fix For: 2.0.0, 1.4.1, 1.5.0, 1.3.3
>
> Attachments: HBASE-19468-poc.patch, HBASE-19468_1.4.patch, 
> HBASE-19468_master.patch
>
>
> We see FNFE exceptions on our 1.3 clusters when scans and flushes happen at 
> the same time. This causes regionserver to throw a UnknownScannerException 
> and client retries.
> This happens during the following sequence:
> 1. Scanner open, client fetched some rows from regionserver and working on it
> 2. Flush happens and storeScanner is updated with flushed files 
> (StoreScanner.updateReaders())
> 3. Compaction happens on the region while scanner is still open
> 4. compaction discharger runs and cleans up the newly flushed file as we 
> don't have new scanners on it yet.
> 5. Client issues scan.next and during StoreScanner.resetScannerStack(), we 
> get a FNFE. RegionServer throws a UnknownScannerThe client retries in 1.3. 
> With branch-1.4, the scan fails with a DoNotRetryIOException.
> [~ram_krish], My proposal is to increment the reader count during 
> updateReaders() and decrement it during resetScannerStack(), so discharger 
> doesn't clean it up. Scan lease expiries also have to be taken care of. Am I 
> missing anything? Is there a better approach?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19525) RS side changes for moving peer modification from zk watcher to procedure

[Duo Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297766#comment-16297766
 ] 

Duo Zhang commented on HBASE-19525:
---

[~openinx] The DummyModifyPeerProcedure and its UT can be removed now. And 
since HBASE-19536 has been committed, you can rebase your patch and then the 
old UTs will work properly?

And please upload the patch to review board.

Thanks.

> RS side changes for moving peer modification from zk watcher to procedure
> -
>
> Key: HBASE-19525
> URL: https://issues.apache.org/jira/browse/HBASE-19525
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2, Replication
>Reporter: Duo Zhang
>Assignee: Zheng Hu
> Fix For: HBASE-19397
>
> Attachments: HBASE-19525.HBASE-19397.v1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19148) Reevaluate default values of configurations

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297760#comment-16297760
 ] 

stack commented on HBASE-19148:
---

Played with setting SNAPPY on by default but was reminded that it could wreck 
havoc if the native lib was not available...

Caused by: java.lang.RuntimeException: native snappy library not available: 
this version of libhadoop was built without snappy support.

... so will skip on this one.

> Reevaluate default values of configurations
> ---
>
> Key: HBASE-19148
> URL: https://issues.apache.org/jira/browse/HBASE-19148
> Project: HBase
>  Issue Type: Bug
>  Components: defaults
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0002-HBASE-19148-Reevaluate-default-values-of-configurati.patch, 
> HBASE-19148.master.001.patch, HBASE-19148.master.002.patch, 
> HBASE-19148.master.003.patch
>
>
> Remove cruft and mythologies. Make descriptions more digestible. Change 
> defaults given experience.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19536) Client side changes for moving peer modification from zk watcher to procedure

[Duo Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-19536:
--
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: HBASE-19397
   Status: Resolved  (was: Patch Available)

Rebased to the newest master and pushed to branch HBASE-19397.

Thanks [~zghaobac] for the contributing.

> Client side changes for moving peer modification from zk watcher to procedure
> -
>
> Key: HBASE-19536
> URL: https://issues.apache.org/jira/browse/HBASE-19536
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Duo Zhang
>Assignee: Guanghao Zhang
> Fix For: HBASE-19397
>
> Attachments: HBASE-19536.HBASE-19397.001.patch, 
> HBASE-19536.HBASE-19397.002.patch, HBASE-19536.HBASE-19397.003.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19536) Client side changes for moving peer modification from zk watcher to procedure

[Duo Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297744#comment-16297744
 ] 

Duo Zhang commented on HBASE-19536:
---

Seems not, HBASE-19558 is opened for it. Let me do a rebase and then commit.

> Client side changes for moving peer modification from zk watcher to procedure
> -
>
> Key: HBASE-19536
> URL: https://issues.apache.org/jira/browse/HBASE-19536
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Duo Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-19536.HBASE-19397.001.patch, 
> HBASE-19536.HBASE-19397.002.patch, HBASE-19536.HBASE-19397.003.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-10092) Move to slf4j

[Appy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Appy updated HBASE-10092:
-
Summary: Move to slf4j  (was: Move up on to log4j2)

> Move to slf4j
> -
>
> Key: HBASE-10092
> URL: https://issues.apache.org/jira/browse/HBASE-10092
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Assignee: Balazs Meszaros
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 10092.txt, 10092v2.txt, HBASE-10092-preview-v0.patch, 
> HBASE-10092.master.001.patch, HBASE-10092.master.002.patch, 
> HBASE-10092.master.003.patch, HBASE-10092.patch
>
>
> Allows logging with less friction.  See http://logging.apache.org/log4j/2.x/  
> This rather radical transition can be done w/ minor change given they have an 
> adapter for apache's logging, the one we use.  They also have and adapter for 
> slf4j so we likely can remove at least some of the 4 versions of this module 
> our dependencies make use of.
> I made a start in attached patch but am currently stuck in maven dependency 
> resolve hell courtesy of our slf4j.  Fixing will take some concentration and 
> a good net connection, an item I currently lack.  Other TODOs are that will 
> need to fix our little log level setting jsp page -- will likely have to undo 
> our use of hadoop's tool here -- and the config system changes a little.
> I will return to this project soon.  Will bring numbers.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-10092) Move up on to log4j2

[Appy (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297716#comment-16297716
 ] 

Appy commented on HBASE-10092:
--

+1
This has tackled many issues already. Let's get it in and work incrementally. 
Great stuff [~balazs.meszaros]. Impressed. :)

On some of the previous things:
- Yeah i meant removing of the guards. That's one thing slf4j was promising. 
But ignore it, too onerous and mechanical of a task to be done by single person
- bq.I did some verification, setting log levels still works through web ui.
Nice! forgot that one.

Just listing down remaining stuff for clarity (so doesn't get lost in extensive 
discussion above):
- Update to log4j 2 backend
- Fix transitive include from hadoop

> Move up on to log4j2
> 
>
> Key: HBASE-10092
> URL: https://issues.apache.org/jira/browse/HBASE-10092
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Assignee: Balazs Meszaros
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 10092.txt, 10092v2.txt, HBASE-10092-preview-v0.patch, 
> HBASE-10092.master.001.patch, HBASE-10092.master.002.patch, 
> HBASE-10092.master.003.patch, HBASE-10092.patch
>
>
> Allows logging with less friction.  See http://logging.apache.org/log4j/2.x/  
> This rather radical transition can be done w/ minor change given they have an 
> adapter for apache's logging, the one we use.  They also have and adapter for 
> slf4j so we likely can remove at least some of the 4 versions of this module 
> our dependencies make use of.
> I made a start in attached patch but am currently stuck in maven dependency 
> resolve hell courtesy of our slf4j.  Fixing will take some concentration and 
> a good net connection, an item I currently lack.  Other TODOs are that will 
> need to fix our little log level setting jsp page -- will likely have to undo 
> our use of hadoop's tool here -- and the config system changes a little.
> I will return to this project soon.  Will bring numbers.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19540) Reduce number of unnecessary semicolons

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297707#comment-16297707
 ] 

Hudson commented on HBASE-19540:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4256 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/4256/])
HBASE-19540 Reduced number of unnecessary semicolons (jan.hentschel: rev 
f46a6d1637bc2c87001f08472e7ae6e39c8caeba)
* (edit) 
hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestWithCellVisibilityLoadAndVerify.java
* (edit) 
hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestProcedureToString.java
* (edit) 
hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/io/MetricsIOSourceImpl.java
* (edit) 
hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/TestStateMachineProcedure.java
* (edit) 
hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java
* (edit) 
hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
* (edit) 
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestIngestWithMOB.java
* (edit) 
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestBackupRestore.java
* (edit) 
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java
* (edit) 
hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/model/ScannerModel.java
* (edit) 
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/SyncTable.java
* (edit) 
hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java
* (edit) 
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
* (edit) 
hbase-it/src/test/java/org/apache/hadoop/hbase/StripeCompactionsPerformanceEvaluation.java
* (edit) 
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/RollingBatchRestartRsExceptMetaAction.java
* (edit) 
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
* (edit) 
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestCopyTable.java


> Reduce number of unnecessary semicolons
> ---
>
> Key: HBASE-19540
> URL: https://issues.apache.org/jira/browse/HBASE-19540
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-19540.master.001.patch, 
> HBASE-19540.master.002.patch
>
>
> Some modules contain unnecessary semicolons, which should be removed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19480) Enable Checkstyle in hbase-annotations

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297712#comment-16297712
 ] 

Hudson commented on HBASE-19480:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4256 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/4256/])
HBASE-19480 Enabled Checkstyle to fail on violations in (jan.hentschel: rev 
df351e4035df0f8db56245ea9cf54952874294ca)
* (edit) hbase-annotations/pom.xml


> Enable Checkstyle in hbase-annotations
> --
>
> Key: HBASE-19480
> URL: https://issues.apache.org/jira/browse/HBASE-19480
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Trivial
> Fix For: 3.0.0, 2.0.0-beta-1
>
> Attachments: HBASE-19480.master.001.patch
>
>
> After all Checkstyle errors are now resolved in the *hbase-annotations* 
> module, Checkstyle can now be configured to fail on violations.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19537) Remove unnecessary semicolons in hbase-backup

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297710#comment-16297710
 ] 

Hudson commented on HBASE-19537:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4256 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/4256/])
HBASE-19537 Removed unnecessary semicolons from hbase-backup (jan.hentschel: 
rev d50ae037160cee874017280147263ce0ab07101a)
* (edit) 
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/IncrementalBackupManager.java
* (edit) 
hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestIncrementalBackupMergeWithFailures.java
* (edit) 
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupSystemTable.java
* (edit) 
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/TableBackupClient.java
* (edit) 
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/RestoreDriver.java
* (edit) 
hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupBase.java
* (edit) 
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/mapreduce/MapReduceBackupCopyJob.java
* (edit) 
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/util/RestoreTool.java
* (edit) 
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupCommands.java
* (edit) 
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/IncrementalTableBackupClient.java
* (edit) 
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupAdminImpl.java
* (edit) 
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/mapreduce/MapReduceBackupMergeJob.java
* (edit) 
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/RestoreTablesClient.java
* (edit) 
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/FullTableBackupClient.java
* (edit) 
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/BackupInfo.java
* (edit) 
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupManifest.java
* (edit) 
hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupDeleteRestore.java


> Remove unnecessary semicolons in hbase-backup
> -
>
> Key: HBASE-19537
> URL: https://issues.apache.org/jira/browse/HBASE-19537
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-19537.master.001.patch
>
>
> Currently *hbase-backup* has some places with unnecessary semicolons, which 
> should be removed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19538) Remove unnecessary semicolons in hbase-client

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297709#comment-16297709
 ] 

Hudson commented on HBASE-19538:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4256 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/4256/])
HBASE-19538 Removed unnecessary semicolons in hbase-client (jan.hentschel: rev 
dd00081c26fb4222e223b510e58923118cf9df43)
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/SnapshotType.java
* (edit) 
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncProcess.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/CompactionState.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/quotas/ThrottlingException.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/quotas/SpaceViolationPolicy.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/ClusterStatus.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/security/SecurityCapability.java


> Remove unnecessary semicolons in hbase-client
> -
>
> Key: HBASE-19538
> URL: https://issues.apache.org/jira/browse/HBASE-19538
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0, 2.0.0-beta-1
>
> Attachments: HBASE-19538.master.001.patch
>
>
> Currently *hbase-client* has some places with unnecessary semicolons, which 
> should be removed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19481) Enable Checkstyle in hbase-error-prone

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297711#comment-16297711
 ] 

Hudson commented on HBASE-19481:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4256 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/4256/])
HBASE-19481 Enabled Checkstyle to fail on violations in (jan.hentschel: rev 
ec7bf57390d8302598901a6e80ca710121cb5f48)
* (edit) hbase-build-support/hbase-error-prone/pom.xml


> Enable Checkstyle in hbase-error-prone
> --
>
> Key: HBASE-19481
> URL: https://issues.apache.org/jira/browse/HBASE-19481
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Trivial
> Fix For: 3.0.0, 1.4.1, 1.5.0, 2.0.0-beta-1
>
> Attachments: BREAKING-HBASE-19481.master.001.patch, 
> HBASE-19481.master.001.patch
>
>
> *hbase-error-prone* doesn't contain any Checkstyle errors. With that 
> Checkstyle can now be configured to fail on violations.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19539) Remove unnecessary semicolons in hbase-common

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297708#comment-16297708
 ] 

Hudson commented on HBASE-19539:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4256 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/4256/])
HBASE-19539 Removed unnecessary semicolons in hbase-common (jan.hentschel: rev 
0f8ea39a5b88b830ca6280e3262e7aa5fcb6280a)
* (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/ClassTestFinder.java
* (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/ClassFinder.java
* (edit) 
hbase-common/src/test/java/org/apache/hadoop/hbase/io/crypto/KeyProviderForTesting.java
* (edit) 
hbase-common/src/main/java/org/apache/hadoop/hbase/io/TagCompressionContext.java
* (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
* (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/ProcedureState.java
* (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/TestClassFinder.java
* (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/TestCellUtil.java
* (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/codec/Codec.java
* (edit) 
hbase-common/src/main/java/org/apache/hadoop/hbase/io/crypto/aes/CommonsCryptoAESDecryptor.java


> Remove unnecessary semicolons in hbase-common
> -
>
> Key: HBASE-19539
> URL: https://issues.apache.org/jira/browse/HBASE-19539
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0, 2.0.0-beta-1
>
> Attachments: HBASE-19539.master.001.patch
>
>
> Currently *hbase-common* has some places with unnecessary semicolons, which 
> should be removed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19555) TestSplitTransactionOnCluster is flaky

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297706#comment-16297706
 ] 

Hudson commented on HBASE-19555:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4256 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/4256/])
HBASE-19555 TestSplitTransactionOnCluster is flaky (tedyu: rev 
dbe409e2c7af5312ab370165b89b57d176486e27)
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java


> TestSplitTransactionOnCluster is flaky
> --
>
> Key: HBASE-19555
> URL: https://issues.apache.org/jira/browse/HBASE-19555
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.0.0-alpha-4
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19555.master.001.patch
>
>
> TestSplitTransactionOnCluster is #3 on flaky list.
> Regions are checked 2 different ways: {{cluster.getRegions(hri.getTable())} 
> vs {{ProtobufUtil.getOnlineRegions(server.getRSRpcServices())}}. The later 
> one can also include {{hbase:namespace}} table as well causing the test to 
> fail.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19148) Reevaluate default values of configurations

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297673#comment-16297673
 ] 

stack commented on HBASE-19148:
---

.003 Set balance by table as per Dave's suggestion. Fix checkstyle and 
whitespace. Nothing on thread asking for new suggestion so let this be what we 
roll with in beta-1.

> Reevaluate default values of configurations
> ---
>
> Key: HBASE-19148
> URL: https://issues.apache.org/jira/browse/HBASE-19148
> Project: HBase
>  Issue Type: Bug
>  Components: defaults
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0002-HBASE-19148-Reevaluate-default-values-of-configurati.patch, 
> HBASE-19148.master.001.patch, HBASE-19148.master.002.patch, 
> HBASE-19148.master.003.patch
>
>
> Remove cruft and mythologies. Make descriptions more digestible. Change 
> defaults given experience.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19148) Reevaluate default values of configurations

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19148:
--
Attachment: HBASE-19148.master.003.patch

> Reevaluate default values of configurations
> ---
>
> Key: HBASE-19148
> URL: https://issues.apache.org/jira/browse/HBASE-19148
> Project: HBase
>  Issue Type: Bug
>  Components: defaults
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0002-HBASE-19148-Reevaluate-default-values-of-configurati.patch, 
> HBASE-19148.master.001.patch, HBASE-19148.master.002.patch, 
> HBASE-19148.master.003.patch
>
>
> Remove cruft and mythologies. Make descriptions more digestible. Change 
> defaults given experience.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-19558) TestRegionsOnMasterOptions hack so it works reliablly

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-19558.
---
Resolution: Fixed

I pushed the below which just waits until there are not RIT before it calls 
cluster shutdown


commit f80d2e9c8e88d3eaeed9a174c770699737f8c574
Author: Michael Stack 
Date:   Tue Dec 19 15:57:11 2017 -0800

HBASE-19558 TestRegionsOnMasterOptions hack so it works reliably

diff --git 
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestRegionsOnMasterOptions.java
 
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestRegionsOnMasterOptions.java
index 7319820d39..58c1db 100644
--- 
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestRegionsOnMasterOptions.java
+++ 
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestRegionsOnMasterOptions.java
@@ -193,6 +193,15 @@ public class TestRegionsOnMasterOptions {
 // still only carry system regions post crash.
 assertEquals(masterCount, mNewActualCount);
   }
+  // Disable balancer and wait till RIT done else cluster won't go down.
+  TEST_UTIL.getAdmin().balancerSwitch(false, true);
+  while (true) {
+if (!TEST_UTIL.getHBaseCluster().getMaster().getAssignmentManager().
+isMetaRegionInTransition()) {
+  break;
+}
+Threads.sleep(10);
+  }
 } finally {
   LOG.info("Running shutdown of cluster");
   TEST_UTIL.shutdownMiniCluster();

> TestRegionsOnMasterOptions hack so it works reliablly
> -
>
> Key: HBASE-19558
> URL: https://issues.apache.org/jira/browse/HBASE-19558
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
>
> See tail of HBASE-19533 for description of problem; our shutdown of a cluster 
> needs work. Meantime hack the test so it doesn't hang on close so it passes 
> reliably again.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19558) TestRegionsOnMasterOptions hack so it works reliablly

[stack (JIRA)]

stack created HBASE-19558:
-

 Summary: TestRegionsOnMasterOptions hack so it works reliablly
 Key: HBASE-19558
 URL: https://issues.apache.org/jira/browse/HBASE-19558
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
Assignee: stack
 Fix For: 2.0.0-beta-1


See tail of HBASE-19533 for description of problem; our shutdown of a cluster 
needs work. Meantime hack the test so it doesn't hang on close so it passes 
reliably again.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19533) How to do controlled shutdown in branch-2?

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297657#comment-16297657
 ] 

stack commented on HBASE-19533:
---

Let me schedule this one. There's a little prob. in shutdown we'll have to fix.

I noticed it over in TestRegionsOnMasterOptions. It fails occasionally. All 
tests pass but for the last one in the list, testRegionsOnAllServers, 
concurrent w/ the shutdown of the cluster, the balancer happens to run and 
schedules a region move. In the unassign of move, all is fine until the update 
of hbase:meta w/ changed state. hbase:meta has just closed so we are stuck 
trying to reach a server that won't come back. We'll retry up to the max. 
Meantime we block the shutdown of the Master. The running thread in the hbase 
client is actually a daemon thread. Its the retrying thread holding a lock that 
prevents the shutdown.

Will be back here.

To reliably reproduce, remove this bit I just added to 
TestRegionsOnMasterOptions:


  // Disable balancer and wait till RIT done else cluster won't go down.
  TEST_UTIL.getAdmin().balancerSwitch(false, true);
  while (true) {
if (!TEST_UTIL.getHBaseCluster().getMaster().getAssignmentManager().
isMetaRegionInTransition()) {
  break;
}
Threads.sleep(10);
  }

> How to do controlled shutdown in branch-2?
> --
>
> Key: HBASE-19533
> URL: https://issues.apache.org/jira/browse/HBASE-19533
> Project: HBase
>  Issue Type: Task
>Reporter: stack
> Fix For: 2.0.0-beta-2
>
>
> Before HBASE-18946, setting shutdown of a cluster, the Master would exit 
> immediately. RegionServers would run region closes and then try and notify 
> the Master of the close and would spew exceptions that the Master was 
> unreachable.
> This is different to how branch-1 used to do it. It used to keep Master up 
> and it would be like the captain of the ship, the last to go down. As of 
> HBASE-18946, this is again the case but there are still open issues.
>  # Usually Master does all open and close of regions. On cluster shutdown, it 
> is the one time where the Regions run the region close. Currently, the 
> regions report the close to the Master which disregards the message since it 
> did not start the region closes. Should we do different? Try and update state 
> in hbase:meta setting it to CLOSE? We might not be able to write CLOSE for 
> all regions since hbase:meta will be closing too (the RS that is hosting 
> hbase:meta will close it last but that may not be enough).
>  # Should the Master run the cluster shutdown sending out close for all 
> regions? What if cluster of 1M regions? Untenable? Send a message per server? 
> That might be better.
> Anyways, this needs attention. Filing issue in meantime.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-15482) Provide an option to skip calculating block locations for SnapshotInputFormat

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-15482:
---
 Hadoop Flags: Reviewed
Fix Version/s: (was: 2.1.0)
   2.0.0-beta-1

Integrated to branch-2 and master.

If you want to backport to branch-1, please attach patch.

Thanks for the patch, Xiang.

Thanks for the review, Jerry.

> Provide an option to skip calculating block locations for SnapshotInputFormat
> -
>
> Key: HBASE-15482
> URL: https://issues.apache.org/jira/browse/HBASE-15482
> Project: HBase
>  Issue Type: Improvement
>  Components: mapreduce
>Reporter: Liyin Tang
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 2.0.0-beta-1
>
> Attachments: 15482.v3.txt, HBASE-15482.master.000.patch, 
> HBASE-15482.master.001.patch, HBASE-15482.master.002.patch, 
> HBASE-15482.master.003.patch
>
>
> When a MR job is reading from SnapshotInputFormat, it needs to calculate the 
> splits based on the block locations in order to get best locality. However, 
> this process may take a long time for large snapshots. 
> In some setup, the computing layer, Spark, Hive or Presto could run out side 
> of HBase cluster. In these scenarios, the block locality doesn't matter. 
> Therefore, it will be great to have an option to skip calculating the block 
> locations for every job. That will super useful for the Hive/Presto/Spark 
> connectors.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19533) How to do controlled shutdown in branch-2?

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19533:
--
Fix Version/s: 2.0.0-beta-2

> How to do controlled shutdown in branch-2?
> --
>
> Key: HBASE-19533
> URL: https://issues.apache.org/jira/browse/HBASE-19533
> Project: HBase
>  Issue Type: Task
>Reporter: stack
> Fix For: 2.0.0-beta-2
>
>
> Before HBASE-18946, setting shutdown of a cluster, the Master would exit 
> immediately. RegionServers would run region closes and then try and notify 
> the Master of the close and would spew exceptions that the Master was 
> unreachable.
> This is different to how branch-1 used to do it. It used to keep Master up 
> and it would be like the captain of the ship, the last to go down. As of 
> HBASE-18946, this is again the case but there are still open issues.
>  # Usually Master does all open and close of regions. On cluster shutdown, it 
> is the one time where the Regions run the region close. Currently, the 
> regions report the close to the Master which disregards the message since it 
> did not start the region closes. Should we do different? Try and update state 
> in hbase:meta setting it to CLOSE? We might not be able to write CLOSE for 
> all regions since hbase:meta will be closing too (the RS that is hosting 
> hbase:meta will close it last but that may not be enough).
>  # Should the Master run the cluster shutdown sending out close for all 
> regions? What if cluster of 1M regions? Untenable? Send a message per server? 
> That might be better.
> Anyways, this needs attention. Filing issue in meantime.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19557) Build and release source jars for hbase-shaded-client and others

[Jerry He (JIRA)]

Jerry He created HBASE-19557:


 Summary: Build and release source jars for hbase-shaded-client and 
others
 Key: HBASE-19557
 URL: https://issues.apache.org/jira/browse/HBASE-19557
 Project: HBase
  Issue Type: Improvement
Affects Versions: 1.2.6, 1.3.1
Reporter: Jerry He


It seems that currently we don't build and release source jars for 
hbase-shaded-client (and server or mapreduce).  IDEs will complain from the 
dependent users. We should provide them.

http://central.maven.org/maven2/org/apache/hbase/hbase-shaded-client/1.3.1/



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19491) Exclude flaky tests from nightly master run

[Appy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Appy updated HBASE-19491:
-
Description: 
Testing infra improvements
- Exclude flaky tests from nightly master run
- Fixes findbugs check (seems like wasn't working earlier : "0  findbugs
0m 0s   Findbugs executables are not available.")
- Add nightly testing against hadoop3


  was:
I was of the opinion that nightly master runs were excluding flaky tests after 
seeing 
https://github.com/apache/hbase/blob/856ee283faf003404e8925006ce0e591c4eba600/dev-support/Jenkinsfile#L54
 few days ago.
After looking at our new set of scripts again and better understanding them, 
looks like that's not enough.
We need to set 
{code}DOCKER_EXTRAARGS=--env=EXCLUDE_TESTS_URL=${EXCLUDE_TESTS_URL}{code} like 
in https://builds.apache.org/job/PreCommit-HBASE-Build/configure


> Exclude flaky tests from nightly master run
> ---
>
> Key: HBASE-19491
> URL: https://issues.apache.org/jira/browse/HBASE-19491
> Project: HBase
>  Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
>
> Testing infra improvements
> - Exclude flaky tests from nightly master run
> - Fixes findbugs check (seems like wasn't working earlier : "0
> findbugs0m 0s   Findbugs executables are not available.")
> - Add nightly testing against hadoop3



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17436) Add facility to provide more information for Other Regions seen on Master UI

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297546#comment-16297546
 ] 

Ted Yu commented on HBASE-17436:


[~stack]:
Do you have other comment ?

> Add facility to provide more information for Other Regions seen on Master UI
> 
>
> Key: HBASE-17436
> URL: https://issues.apache.org/jira/browse/HBASE-17436
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Janos Gub
>  Labels: ui, usability
> Fix For: 2.0.0
>
> Attachments: 17436.v10.txt, 17436.v11.txt, 17436.v9.txt, 
> HBASE-17436-v2.patch, HBASE-17436-v5.patch, HBASE-17436-v6.patch, 
> HBASE-17436-v6.patch, HBASE-17436-v7.patch, HBASE-17436-v8.patch, 
> HBASE-17436.patch, HBASE-17779-v3.patch, HBASE-17779-v4.patch, Screen Shot 
> 2017-04-24 at 8.43.31 PM.png, Screen Shot 2017-04-26 at 4.39.49 PM.png, 
> Screen Shot 2017-11-08 at 10.29.49 AM.png, Screen Shot 2017-11-08 at 10.30.09 
> AM.png, Screen Shot 2017-11-22 at 4.17.35 PM.png, initial.patch, 
> regionstates.png, table-selected.png, tableregions.png
>
>
> [~rpednekar] and I were looking at a case where the count displayed under 
> Other Regions was high (~1200).
> Since the table page just maintains a count instead of List of region names, 
> it is very difficult for user to determine which regions belong to this 
> category.
> We should provide facility to provide more details for this category (LOG, 
> JMX, etc).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-10092) Move up on to log4j2

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297519#comment-16297519
 ] 

stack commented on HBASE-10092:
---

I tried it.

We'll have to fix transitive include from hadoop... Can do in follow-on

stack@ve0524:~$ ./hbase/bin/start-hbase.sh --config ~/conf_hbase
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in 
[jar:file:/home/stack/hbase-3.0.0-SNAPSHOT/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in 
[jar:file:/home/stack/hadoop-3.0.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in 
[jar:file:/home/stack/hbase-3.0.0-SNAPSHOT/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in 
[jar:file:/home/stack/hadoop-3.0.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
ve0524.halxg.cloudera.com: running zookeeper, logging to 
/home/stack/logs/hbase-stack-zookeeper-ve0524.out
running master, logging to /home/stack/logs/hbase-stack-master-ve0524.out

Log format is the same (duh)

Setting log level works for me (hbase master branch on hadoop3)

This move to slf4j should be a subissue of this one? We are still on log4j 
backend.

Related, useful blog benchmarking logging frameworks 
https://www.loggly.com/blog/benchmarking-java-logging-frameworks/





> Move up on to log4j2
> 
>
> Key: HBASE-10092
> URL: https://issues.apache.org/jira/browse/HBASE-10092
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Assignee: Balazs Meszaros
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 10092.txt, 10092v2.txt, HBASE-10092-preview-v0.patch, 
> HBASE-10092.master.001.patch, HBASE-10092.master.002.patch, 
> HBASE-10092.master.003.patch, HBASE-10092.patch
>
>
> Allows logging with less friction.  See http://logging.apache.org/log4j/2.x/  
> This rather radical transition can be done w/ minor change given they have an 
> adapter for apache's logging, the one we use.  They also have and adapter for 
> slf4j so we likely can remove at least some of the 4 versions of this module 
> our dependencies make use of.
> I made a start in attached patch but am currently stuck in maven dependency 
> resolve hell courtesy of our slf4j.  Fixing will take some concentration and 
> a good net connection, an item I currently lack.  Other TODOs are that will 
> need to fix our little log level setting jsp page -- will likely have to undo 
> our use of hadoop's tool here -- and the config system changes a little.
> I will return to this project soon.  Will bring numbers.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19171) Update package references to match new shaded offset in hbase-thirdparty

[Mike Drob (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297517#comment-16297517
 ] 

Mike Drob commented on HBASE-19171:
---

Didn't realize this issue existed, thanks [~jinghe]!

> Update package references to match new shaded offset in hbase-thirdparty
> 
>
> Key: HBASE-19171
> URL: https://issues.apache.org/jira/browse/HBASE-19171
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Jerry He
>Priority: Critical
>
> This has dependency on the parent task, and can only be done after a new 
> hbase-thirdparty release.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19552) update hbase to use new thirdparty libs

[Mike Drob (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob updated HBASE-19552:
--
Fix Version/s: 2.0.0-beta-1

> update hbase to use new thirdparty libs
> ---
>
> Key: HBASE-19552
> URL: https://issues.apache.org/jira/browse/HBASE-19552
> Project: HBase
>  Issue Type: Task
>  Components: dependencies, thirdparty
>Reporter: Mike Drob
>Assignee: Mike Drob
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19552.patch, HBASE-19552.v2.patch
>
>
> When we release hbase-thirdparty next release, we will need to update our 
> code for the packaging change.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19556) Remove TestAssignmentManager#testGoodSplit, which no longer make sense

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297494#comment-16297494
 ] 

Hadoop QA commented on HBASE-19556:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
 3s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
13s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
17m 25s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}100m 26s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
20s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}135m 18s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19556 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12902919/HBASE-19556-master-v1.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 306c9ee1b190 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 0f8ea39a5b |
| maven | version: Apache Maven 3.5.2 
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10569/artifact/patchprocess/patch-unit-hbase-server.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10569/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10569/console |
| Powered by | Apache Yetus 0.6.0   http://yetus.apache.org |


This message was automatically generated.



> Remove TestAssignmentManager#testGoodSplit, which no longer make sense
> 

[jira] [Updated] (HBASE-19556) Remove TestAssignmentManager#testGoodSplit, which no longer make sense

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19556:
--
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Pushed to master and branch-2. Thanks for patch [~easyliangjob]

> Remove TestAssignmentManager#testGoodSplit, which no longer make sense
> --
>
> Key: HBASE-19556
> URL: https://issues.apache.org/jira/browse/HBASE-19556
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 2.0.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19556-master-v1.patch
>
>
> {quote}
> [ERROR] 
> testGoodSplit(org.apache.hadoop.hbase.master.assignment.TestAssignmentManager)
>   Time elapsed: 0.478 s  <<< ERROR!
> java.io.IOException: 5a50732f7cb3a05dd3a297bacbc34943 NOT splittable
> {quote}
> GoodSplitExecutor can only mock some functions in RSProcedureDispatcher, 
> however, to test split, we need to create(or mock) a real table on region 
> server side. And GoodSplitExecutor can not mock those kinds of function.
> And, similar test has already been covered in TestSplitTableRegionProcedure, 
> we no longer need to test it again in TestAssignmentManager



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19494) Create simple WALKey filter that can be plugged in on the Replication Sink

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19494:
--
Release Note: 
Adds means of adding very basic filter on the sink side of replication. We 
already have a means of installing filter source-side, which is better place to 
filter edits before they are shipped over the network, but this facility is 
needed by hbase-indexer.

Set hbase.replication.sink.walentrysinkfilter with a no-param Constructor 
implementation. See test in patch for example.

> Create simple WALKey filter that can be plugged in on the Replication Sink
> --
>
> Key: HBASE-19494
> URL: https://issues.apache.org/jira/browse/HBASE-19494
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0003-HBASE-19494-Create-simple-WALKey-filter-that-can-be-.patch, 
> HBASE-19494.master.001.patch
>
>
> hbase-indexer used to look at WALKeys on the sink to see if their time of 
> creation was before the time at which the replication stream was enabled.
> In the parent redo, there is no means for doing this anymore (because WALKey 
> used to be Private and because to get at the WALKey in the Sink, you had to 
> override all of the Replication which meant importing a million Private 
> objects...).
> This issue is about adding a simple filter to Replication on the sink-side 
> that just takes a WALKey (now InterfaceAudience LimitedPrivate and recently 
> made read-only).
> Assigned myself. Need to do this so hbase-indexer can move to hbase2.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19552) update hbase to use new thirdparty libs

[Jerry He (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297487#comment-16297487
 ] 

Jerry He commented on HBASE-19552:
--

Glanced through.  Looks good.

> update hbase to use new thirdparty libs
> ---
>
> Key: HBASE-19552
> URL: https://issues.apache.org/jira/browse/HBASE-19552
> Project: HBase
>  Issue Type: Task
>  Components: dependencies, thirdparty
>Reporter: Mike Drob
>Assignee: Mike Drob
> Attachments: HBASE-19552.patch, HBASE-19552.v2.patch
>
>
> When we release hbase-thirdparty next release, we will need to update our 
> code for the packaging change.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19494) Create simple WALKey filter that can be plugged in on the Replication Sink

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19494:
--
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Pushed to branch-2 and to master. Thanks for reviews [~whoschek]

> Create simple WALKey filter that can be plugged in on the Replication Sink
> --
>
> Key: HBASE-19494
> URL: https://issues.apache.org/jira/browse/HBASE-19494
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0003-HBASE-19494-Create-simple-WALKey-filter-that-can-be-.patch, 
> HBASE-19494.master.001.patch
>
>
> hbase-indexer used to look at WALKeys on the sink to see if their time of 
> creation was before the time at which the replication stream was enabled.
> In the parent redo, there is no means for doing this anymore (because WALKey 
> used to be Private and because to get at the WALKey in the Sink, you had to 
> override all of the Replication which meant importing a million Private 
> objects...).
> This issue is about adding a simple filter to Replication on the sink-side 
> that just takes a WALKey (now InterfaceAudience LimitedPrivate and recently 
> made read-only).
> Assigned myself. Need to do this so hbase-indexer can move to hbase2.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-19171) Update package references to match new shaded offset in hbase-thirdparty

[Jerry He (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jerry He resolved HBASE-19171.
--
   Resolution: Duplicate
Fix Version/s: (was: 2.0.0-beta-1)

Resolve as dup of the other task [~mdrob] is working on.

> Update package references to match new shaded offset in hbase-thirdparty
> 
>
> Key: HBASE-19171
> URL: https://issues.apache.org/jira/browse/HBASE-19171
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Jerry He
>Priority: Critical
>
> This has dependency on the parent task, and can only be done after a new 
> hbase-thirdparty release.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19494) Create simple WALKey filter that can be plugged in on the Replication Sink

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297481#comment-16297481
 ] 

stack commented on HBASE-19494:
---

We have a few timeouts going on. This time it was this:

[INFO] Results:
[INFO] 
[WARNING] Flakes: 
[WARNING] 
org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient.testSkipFlushTableSnapshot(org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient)
[ERROR]   Run 1: TestFlushSnapshotFromClient.testSkipFlushTableSnapshot:201 
Expected snapshot to contain family 'fam', but it does not.
[INFO]   Run 2: PASS
[INFO] 
[INFO] 
[WARNING] Tests run: 3402, Failures: 0, Errors: 0, Skipped: 33, Flakes: 1


Need to fix.

Unrelated. Let me push this change.

> Create simple WALKey filter that can be plugged in on the Replication Sink
> --
>
> Key: HBASE-19494
> URL: https://issues.apache.org/jira/browse/HBASE-19494
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0003-HBASE-19494-Create-simple-WALKey-filter-that-can-be-.patch, 
> HBASE-19494.master.001.patch
>
>
> hbase-indexer used to look at WALKeys on the sink to see if their time of 
> creation was before the time at which the replication stream was enabled.
> In the parent redo, there is no means for doing this anymore (because WALKey 
> used to be Private and because to get at the WALKey in the Sink, you had to 
> override all of the Replication which meant importing a million Private 
> objects...).
> This issue is about adding a simple filter to Replication on the sink-side 
> that just takes a WALKey (now InterfaceAudience LimitedPrivate and recently 
> made read-only).
> Assigned myself. Need to do this so hbase-indexer can move to hbase2.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-15482) Provide an option to skip calculating block locations for SnapshotInputFormat

[Jerry He (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297478#comment-16297478
 ] 

Jerry He commented on HBASE-15482:
--

+1 on the latest patch.

> Provide an option to skip calculating block locations for SnapshotInputFormat
> -
>
> Key: HBASE-15482
> URL: https://issues.apache.org/jira/browse/HBASE-15482
> Project: HBase
>  Issue Type: Improvement
>  Components: mapreduce
>Reporter: Liyin Tang
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 2.1.0
>
> Attachments: 15482.v3.txt, HBASE-15482.master.000.patch, 
> HBASE-15482.master.001.patch, HBASE-15482.master.002.patch, 
> HBASE-15482.master.003.patch
>
>
> When a MR job is reading from SnapshotInputFormat, it needs to calculate the 
> splits based on the block locations in order to get best locality. However, 
> this process may take a long time for large snapshots. 
> In some setup, the computing layer, Spark, Hive or Presto could run out side 
> of HBase cluster. In these scenarios, the block locality doesn't matter. 
> Therefore, it will be great to have an option to skip calculating the block 
> locations for every job. That will super useful for the Hive/Presto/Spark 
> connectors.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-15482) Provide an option to skip calculating block locations for SnapshotInputFormat

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297466#comment-16297466
 ] 

Ted Yu commented on HBASE-15482:


lgtm

> Provide an option to skip calculating block locations for SnapshotInputFormat
> -
>
> Key: HBASE-15482
> URL: https://issues.apache.org/jira/browse/HBASE-15482
> Project: HBase
>  Issue Type: Improvement
>  Components: mapreduce
>Reporter: Liyin Tang
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 2.1.0
>
> Attachments: 15482.v3.txt, HBASE-15482.master.000.patch, 
> HBASE-15482.master.001.patch, HBASE-15482.master.002.patch, 
> HBASE-15482.master.003.patch
>
>
> When a MR job is reading from SnapshotInputFormat, it needs to calculate the 
> splits based on the block locations in order to get best locality. However, 
> this process may take a long time for large snapshots. 
> In some setup, the computing layer, Spark, Hive or Presto could run out side 
> of HBase cluster. In these scenarios, the block locality doesn't matter. 
> Therefore, it will be great to have an option to skip calculating the block 
> locations for every job. That will super useful for the Hive/Presto/Spark 
> connectors.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19494) Create simple WALKey filter that can be plugged in on the Replication Sink

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297459#comment-16297459
 ] 

Hadoop QA commented on HBASE-19494:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  
6s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
 3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
36s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  1m  
3s{color} | {color:red} hbase-server: The patch generated 22 new + 17 unchanged 
- 3 fixed = 39 total (was 20) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
34s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
18m 49s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}113m 41s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
18s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}151m  5s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19494 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12902907/HBASE-19494.master.001.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 9e6be9603a28 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11 
12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / dbe409e2c7 |
| maven | version: Apache Maven 3.5.2 
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10567/artifact/patchprocess/diff-checkstyle-hbase-server.txt
 |
| unit | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10567/artifact/patchprocess/patch-unit-hbase-server.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10567/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/10567/console |
| Powered by | Apache Yetus 

[jira] [Commented] (HBASE-10092) Move up on to log4j2

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297431#comment-16297431
 ] 

stack commented on HBASE-10092:
---

Thanks [~mikael.stal...@home.se]

Lets commit this. Going to log4j2 we can do in a follow-on, later.

> Move up on to log4j2
> 
>
> Key: HBASE-10092
> URL: https://issues.apache.org/jira/browse/HBASE-10092
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Assignee: Balazs Meszaros
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 10092.txt, 10092v2.txt, HBASE-10092-preview-v0.patch, 
> HBASE-10092.master.001.patch, HBASE-10092.master.002.patch, 
> HBASE-10092.master.003.patch, HBASE-10092.patch
>
>
> Allows logging with less friction.  See http://logging.apache.org/log4j/2.x/  
> This rather radical transition can be done w/ minor change given they have an 
> adapter for apache's logging, the one we use.  They also have and adapter for 
> slf4j so we likely can remove at least some of the 4 versions of this module 
> our dependencies make use of.
> I made a start in attached patch but am currently stuck in maven dependency 
> resolve hell courtesy of our slf4j.  Fixing will take some concentration and 
> a good net connection, an item I currently lack.  Other TODOs are that will 
> need to fix our little log level setting jsp page -- will likely have to undo 
> our use of hadoop's tool here -- and the config system changes a little.
> I will return to this project soon.  Will bring numbers.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-10092) Move up on to log4j2

[JIRA]

[ 
https://issues.apache.org/jira/browse/HBASE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297413#comment-16297413
 ] 

Mikael Ståldal commented on HBASE-10092:


https://www.slf4j.org/faq.html#fatal is not correct, you can do that with Log4j 
2 as well.

You can use SLF4J as frontend with Log4j 2 as backend.

However, then you cannot use Log4j 1 configuration files. As far as I know, 
only Log4j 1 can read those configuration files.

> Move up on to log4j2
> 
>
> Key: HBASE-10092
> URL: https://issues.apache.org/jira/browse/HBASE-10092
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Assignee: Balazs Meszaros
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 10092.txt, 10092v2.txt, HBASE-10092-preview-v0.patch, 
> HBASE-10092.master.001.patch, HBASE-10092.master.002.patch, 
> HBASE-10092.master.003.patch, HBASE-10092.patch
>
>
> Allows logging with less friction.  See http://logging.apache.org/log4j/2.x/  
> This rather radical transition can be done w/ minor change given they have an 
> adapter for apache's logging, the one we use.  They also have and adapter for 
> slf4j so we likely can remove at least some of the 4 versions of this module 
> our dependencies make use of.
> I made a start in attached patch but am currently stuck in maven dependency 
> resolve hell courtesy of our slf4j.  Fixing will take some concentration and 
> a good net connection, an item I currently lack.  Other TODOs are that will 
> need to fix our little log level setting jsp page -- will likely have to undo 
> our use of hadoop's tool here -- and the config system changes a little.
> I will return to this project soon.  Will bring numbers.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19148) Reevaluate default values of configurations

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297411#comment-16297411
 ] 

stack commented on HBASE-19148:
---

Thanks [~anastas] Added link here.

> Reevaluate default values of configurations
> ---
>
> Key: HBASE-19148
> URL: https://issues.apache.org/jira/browse/HBASE-19148
> Project: HBase
>  Issue Type: Bug
>  Components: defaults
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0002-HBASE-19148-Reevaluate-default-values-of-configurati.patch, 
> HBASE-19148.master.001.patch, HBASE-19148.master.002.patch
>
>
> Remove cruft and mythologies. Make descriptions more digestible. Change 
> defaults given experience.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19480) Enable Checkstyle in hbase-annotations

[Jan Hentschel (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Hentschel updated HBASE-19480:
--
   Resolution: Fixed
Fix Version/s: 2.0.0-beta-1
   3.0.0
   Status: Resolved  (was: Patch Available)

> Enable Checkstyle in hbase-annotations
> --
>
> Key: HBASE-19480
> URL: https://issues.apache.org/jira/browse/HBASE-19480
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Trivial
> Fix For: 3.0.0, 2.0.0-beta-1
>
> Attachments: HBASE-19480.master.001.patch
>
>
> After all Checkstyle errors are now resolved in the *hbase-annotations* 
> module, Checkstyle can now be configured to fail on violations.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-10092) Move up on to log4j2

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297407#comment-16297407
 ] 

stack commented on HBASE-10092:
---

The former.

[~mikael.stal...@home.se] SLF4J as frontend reading our log4j1 property files 
with log4j2 as our backend?

> Move up on to log4j2
> 
>
> Key: HBASE-10092
> URL: https://issues.apache.org/jira/browse/HBASE-10092
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Assignee: Balazs Meszaros
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 10092.txt, 10092v2.txt, HBASE-10092-preview-v0.patch, 
> HBASE-10092.master.001.patch, HBASE-10092.master.002.patch, 
> HBASE-10092.master.003.patch, HBASE-10092.patch
>
>
> Allows logging with less friction.  See http://logging.apache.org/log4j/2.x/  
> This rather radical transition can be done w/ minor change given they have an 
> adapter for apache's logging, the one we use.  They also have and adapter for 
> slf4j so we likely can remove at least some of the 4 versions of this module 
> our dependencies make use of.
> I made a start in attached patch but am currently stuck in maven dependency 
> resolve hell courtesy of our slf4j.  Fixing will take some concentration and 
> a good net connection, an item I currently lack.  Other TODOs are that will 
> need to fix our little log level setting jsp page -- will likely have to undo 
> our use of hadoop's tool here -- and the config system changes a little.
> I will return to this project soon.  Will bring numbers.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19494) Create simple WALKey filter that can be plugged in on the Replication Sink

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297392#comment-16297392
 ] 

stack commented on HBASE-19494:
---

Thanks for the feedback [~whoschek]

Will commit when hadoopqa comes back.


> Create simple WALKey filter that can be plugged in on the Replication Sink
> --
>
> Key: HBASE-19494
> URL: https://issues.apache.org/jira/browse/HBASE-19494
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0003-HBASE-19494-Create-simple-WALKey-filter-that-can-be-.patch, 
> HBASE-19494.master.001.patch
>
>
> hbase-indexer used to look at WALKeys on the sink to see if their time of 
> creation was before the time at which the replication stream was enabled.
> In the parent redo, there is no means for doing this anymore (because WALKey 
> used to be Private and because to get at the WALKey in the Sink, you had to 
> override all of the Replication which meant importing a million Private 
> objects...).
> This issue is about adding a simple filter to Replication on the sink-side 
> that just takes a WALKey (now InterfaceAudience LimitedPrivate and recently 
> made read-only).
> Assigned myself. Need to do this so hbase-indexer can move to hbase2.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19481) Enable Checkstyle in hbase-error-prone

[Jan Hentschel (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Hentschel updated HBASE-19481:
--
   Resolution: Fixed
Fix Version/s: 2.0.0-beta-1
   1.5.0
   1.4.1
   3.0.0
   Status: Resolved  (was: Patch Available)

> Enable Checkstyle in hbase-error-prone
> --
>
> Key: HBASE-19481
> URL: https://issues.apache.org/jira/browse/HBASE-19481
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Trivial
> Fix For: 3.0.0, 1.4.1, 1.5.0, 2.0.0-beta-1
>
> Attachments: BREAKING-HBASE-19481.master.001.patch, 
> HBASE-19481.master.001.patch
>
>
> *hbase-error-prone* doesn't contain any Checkstyle errors. With that 
> Checkstyle can now be configured to fail on violations.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-18684) [hbase-thirdparty] Remove CHANGES, update NOTICES

[Mike Drob (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob resolved HBASE-18684.
---
Resolution: Fixed

> [hbase-thirdparty] Remove CHANGES, update NOTICES
> -
>
> Key: HBASE-18684
> URL: https://issues.apache.org/jira/browse/HBASE-18684
> Project: HBase
>  Issue Type: Task
>  Components: hbase-thirdparty
>Reporter: stack
>Assignee: Mike Drob
> Fix For: thirdparty-2.0.0
>
> Attachments: HBASE-18684.patch
>
>
> From [~mdrob] on vote thread for hbase-thirdparty 1.0.1RC0:
> "LICENSE question... do we actually bundle jquery, asciidoctor, and the orca
> logo here? I didn't see them anywhere.
> CHANGES is mostly blank... If we don't use it, get rid of it in the next
> version."
> Let me fix in next release.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19494) Create simple WALKey filter that can be plugged in on the Replication Sink

[wolfgang hoschek (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297391#comment-16297391
 ] 

wolfgang hoschek commented on HBASE-19494:
--

Great. I think this works. Thanks!

> Create simple WALKey filter that can be plugged in on the Replication Sink
> --
>
> Key: HBASE-19494
> URL: https://issues.apache.org/jira/browse/HBASE-19494
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0003-HBASE-19494-Create-simple-WALKey-filter-that-can-be-.patch, 
> HBASE-19494.master.001.patch
>
>
> hbase-indexer used to look at WALKeys on the sink to see if their time of 
> creation was before the time at which the replication stream was enabled.
> In the parent redo, there is no means for doing this anymore (because WALKey 
> used to be Private and because to get at the WALKey in the Sink, you had to 
> override all of the Replication which meant importing a million Private 
> objects...).
> This issue is about adding a simple filter to Replication on the sink-side 
> that just takes a WALKey (now InterfaceAudience LimitedPrivate and recently 
> made read-only).
> Assigned myself. Need to do this so hbase-indexer can move to hbase2.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-10092) Move up on to log4j2

[JIRA]

[ 
https://issues.apache.org/jira/browse/HBASE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297390#comment-16297390
 ] 

Mikael Ståldal commented on HBASE-10092:


{quote}SLF4J doc says it will only show if backend is logback. Otherwise, 
dropped. Good to have it though. We'll have to call out in release notes that 
FATAL no longer shows in our logs (usually){quote}

Are you referring to https://www.slf4j.org/faq.html#fatal or 
https://www.slf4j.org/faq.html#marker_interface ?

That's not correct, Log4j 2 also support markers.


> Move up on to log4j2
> 
>
> Key: HBASE-10092
> URL: https://issues.apache.org/jira/browse/HBASE-10092
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Assignee: Balazs Meszaros
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 10092.txt, 10092v2.txt, HBASE-10092-preview-v0.patch, 
> HBASE-10092.master.001.patch, HBASE-10092.master.002.patch, 
> HBASE-10092.master.003.patch, HBASE-10092.patch
>
>
> Allows logging with less friction.  See http://logging.apache.org/log4j/2.x/  
> This rather radical transition can be done w/ minor change given they have an 
> adapter for apache's logging, the one we use.  They also have and adapter for 
> slf4j so we likely can remove at least some of the 4 versions of this module 
> our dependencies make use of.
> I made a start in attached patch but am currently stuck in maven dependency 
> resolve hell courtesy of our slf4j.  Fixing will take some concentration and 
> a good net connection, an item I currently lack.  Other TODOs are that will 
> need to fix our little log level setting jsp page -- will likely have to undo 
> our use of hadoop's tool here -- and the config system changes a little.
> I will return to this project soon.  Will bring numbers.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-19552) update hbase to use new thirdparty libs

[Mike Drob (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob updated HBASE-19552:
--
Attachment: HBASE-19552.v2.patch

v2: rebased, also dropping the ugly system property from tests for netty 
relocation

> update hbase to use new thirdparty libs
> ---
>
> Key: HBASE-19552
> URL: https://issues.apache.org/jira/browse/HBASE-19552
> Project: HBase
>  Issue Type: Task
>  Components: dependencies, thirdparty
>Reporter: Mike Drob
>Assignee: Mike Drob
> Attachments: HBASE-19552.patch, HBASE-19552.v2.patch
>
>
> When we release hbase-thirdparty next release, we will need to update our 
> code for the packaging change.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18970) The version of jruby we use now can't get interactive input from prompt

[Mike Drob (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297379#comment-16297379
 ] 

Mike Drob commented on HBASE-18970:
---

Neat catch on the stderr redirection, patch LGTM, will try to commit later.

> The version of jruby we use now can't get interactive input from prompt
> ---
>
> Key: HBASE-18970
> URL: https://issues.apache.org/jira/browse/HBASE-18970
> Project: HBase
>  Issue Type: Task
>  Components: shell
>Reporter: Chia-Ping Tsai
>Assignee: AMIT VIRMANI
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: HBase-18970.branch-2.001.patch, samplecmd.txt, 
> test_shell-Output.txt, test_shell.sh
>
>
> case 1: press enter
> {code}
> hbase(main):002:0> disable_all 'chia(.*)'
> chia_1
>   
>
> Disable the above 1 tables (y/n)?
> y^M
> {code}
> case 2: press ctrl-j
> {code}
> hbase(main):001:0> disable_all 'chia(.*)'
> chia_1
>   
>
> Disable the above 1 tables (y/n)?
> y^J1 tables successfully disabled
> Took 5.0059 seconds  
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (HBASE-18970) The version of jruby we use now can't get interactive input from prompt

[Mike Drob (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob reassigned HBASE-18970:
-

Assignee: AMIT VIRMANI  (was: Mike Drob)

> The version of jruby we use now can't get interactive input from prompt
> ---
>
> Key: HBASE-18970
> URL: https://issues.apache.org/jira/browse/HBASE-18970
> Project: HBase
>  Issue Type: Task
>  Components: shell
>Reporter: Chia-Ping Tsai
>Assignee: AMIT VIRMANI
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: HBase-18970.branch-2.001.patch, samplecmd.txt, 
> test_shell-Output.txt, test_shell.sh
>
>
> case 1: press enter
> {code}
> hbase(main):002:0> disable_all 'chia(.*)'
> chia_1
>   
>
> Disable the above 1 tables (y/n)?
> y^M
> {code}
> case 2: press ctrl-j
> {code}
> hbase(main):001:0> disable_all 'chia(.*)'
> chia_1
>   
>
> Disable the above 1 tables (y/n)?
> y^J1 tables successfully disabled
> Took 5.0059 seconds  
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-19556) Remove TestAssignmentManager#testGoodSplit, which no longer make sense

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297376#comment-16297376
 ] 

stack commented on HBASE-19556:
---

Thanks [~easyliangjob] +1

> Remove TestAssignmentManager#testGoodSplit, which no longer make sense
> --
>
> Key: HBASE-19556
> URL: https://issues.apache.org/jira/browse/HBASE-19556
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 2.0.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19556-master-v1.patch
>
>
> {quote}
> [ERROR] 
> testGoodSplit(org.apache.hadoop.hbase.master.assignment.TestAssignmentManager)
>   Time elapsed: 0.478 s  <<< ERROR!
> java.io.IOException: 5a50732f7cb3a05dd3a297bacbc34943 NOT splittable
> {quote}
> GoodSplitExecutor can only mock some functions in RSProcedureDispatcher, 
> however, to test split, we need to create(or mock) a real table on region 
> server side. And GoodSplitExecutor can not mock those kinds of function.
> And, similar test has already been covered in TestSplitTableRegionProcedure, 
> we no longer need to test it again in TestAssignmentManager



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)