[jira] [Commented] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011147#comment-15011147
 ] 

Sean Busbey commented on HBASE-14498:
-

if the problem exists in versions prior to 1.1, please mark them as target 
versions and ensure the fix goes to them.

> Master stuck in infinite loop when all Zookeeper servers are unreachable
> 
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
>  Issue Type: Bug
>  Components: master
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Blocker
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4
>
> Attachments: HBASE-14498-V2.patch, HBASE-14498-V3.patch, 
> HBASE-14498-V4.patch, HBASE-14498.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to 
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO 
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)] 
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in 
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and 
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO 
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)] 
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)] 
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN 
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)] 
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO 
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)] 
> zookeeper.ClientCnxn: Opening socket connection to server 
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown 
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)] 
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)] 
> zookeeper.ClientCnxn: Opening socket connection to server 
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown 
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)] 
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in 
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and 
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000] 
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, 
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181, 
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException: 
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)] 
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)] 
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)] 
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181. 
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt 
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an 
> > active master. 
> > HM2 is keep on waiting for region server to report him as part of active 
> > master intialization.
> {noformat} 
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting 
> for region servers count to settle; currently checked in 0, slept for 0 ms, 
> expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval 
> of 1500 ms. | 
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> ---
> ---
> 2015-09-26 15:32:50,841 | INFO | HM2-Host:21300.activeMasterManager | Waiting 
> for region servers count to settle; currently checked in 0, slept for 483913 
> ms, expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, 
> interval of 1500 ms. | 
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> {noformat}
> > At other end, region servers are reporting to HM1 on 3 sec interval. Here 
> > region server retrieve master location from zookeeper only when they 
> > couldn't connect to Master (ServiceException).
> Region Server will not report HM2 as per current design until unless HM1 
> abort,so HM2 will exit(InitializationMonitor) and again wait for region 

[jira] [Created] (HBASE-14832) Ensure write paths work with ByteBufferedCells in case of compaction

[ramkrishna.s.vasudevan (JIRA)]

ramkrishna.s.vasudevan created HBASE-14832:
--

 Summary: Ensure write paths work with ByteBufferedCells in case of 
compaction
 Key: HBASE-14832
 URL: https://issues.apache.org/jira/browse/HBASE-14832
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 2.0.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Fix For: 2.0.0


Currently any cell coming out of offheap Bucketcache while compaction does a 
copy using the getXXXArray() API since write path does not work with BBCells. 
This JIRA is aimed at changing the write path to support BBCells so that this 
copy is avoided.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14803) Add some debug logs to StoreFileScanner

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011347#comment-15011347
 ] 

stack commented on HBASE-14803:
---

bq. I tried to run the tests locally and it fails on unrelated timeout after 
hours. Let see how it goes here.

What was not working, do you know JMS? I'm interested.

> Add some debug logs to StoreFileScanner
> ---
>
> Key: HBASE-14803
> URL: https://issues.apache.org/jira/browse/HBASE-14803
> Project: HBase
>  Issue Type: Bug
>Reporter: Jean-Marc Spaggiari
>Assignee: Jean-Marc Spaggiari
>Priority: Minor
>  Labels: beginner
> Fix For: 1.2.0
>
> Attachments: HBASE-14803.v0-trunk.patch, HBASE-14803.v1-trunk.patch, 
> HBASE-14803.v2-trunk.patch, HBASE-14803.v3-trunk.patch, 
> HBASE-14803.v4-trunk.patch, HBASE-14803.v4-trunk.patch, 
> HBASE-14803.v5-trunk.patch
>
>
> To validate some behaviors I had to add some logs into StoreFileScanner.
> I think it can be interesting for other people looking for debuging. So 
> sharing the modifications here.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14575) Reduce scope of compactions holding region lock

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011377#comment-15011377
 ] 

Ted Yu commented on HBASE-14575:


The description from 04/Nov/15 09:00 is mostly accurate:
w.r.t. #4 swap in compacted files, region lock is not needed accordingly to 
subsequent discussion.

bq. why is not included in the patch

The explanation is pretty long. How about putting the explanation in 
description of this JIRA and refer to JIRA number in the patch ?

I reran ITBLL with the following parameters and it passed as well (with region 
lock relaxation):

bin/hbase --config /etc/hbase/conf 
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList Loop 4 12 200 
/tmp/hbase-biglinkedlist-verify 8 50 --monkey slowDeterministic


> Reduce scope of compactions holding region lock
> ---
>
> Key: HBASE-14575
> URL: https://issues.apache.org/jira/browse/HBASE-14575
> Project: HBase
>  Issue Type: Sub-task
>  Components: Compaction, regionserver
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: 14575-test-case.patch, 14575-v1.patch, 14575-v2.patch, 
> 14575-v3.patch, 14575-v4.patch, 14575-v5.patch, 14575-v6.txt, 14575-v6.txt, 
> 14575-v6.txt, 14575.v00.patch
>
>
> Per [~devaraj]'s idea on parent issue, let's see if we can reduce the scope 
> of critical section under which compactions hold the region read lock.
> Here is summary from parent issue:
> Another idea is we can reduce the scope of when the read lock is held during 
> compaction. In theory the compactor only needs a region read lock while 
> deciding what files to compact and at the time of committing the compaction. 
> We're protected from the case of region close events because compactions are 
> checking (every 10k bytes written) if the store has been closed in order to 
> abort in such a case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14575) Reduce scope of compactions holding region lock

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011388#comment-15011388
 ] 

stack commented on HBASE-14575:
---

bq. The explanation is pretty long. How about putting the explanation in 
description of this JIRA and refer to JIRA number in the patch ?

Put in the code I'd say. Thats where folks will be looking for description of 
locking regime. Here it will just rot. Can add (with update) on commit.

Your ITBLL numbers are a little odd but fine. Any exceptions in regionserver 
logs?

I'd be game for trying this patch... do it in master and in branch-1 -- not 1.2 
-- so we have time to see if any issues arise.

> Reduce scope of compactions holding region lock
> ---
>
> Key: HBASE-14575
> URL: https://issues.apache.org/jira/browse/HBASE-14575
> Project: HBase
>  Issue Type: Sub-task
>  Components: Compaction, regionserver
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: 14575-test-case.patch, 14575-v1.patch, 14575-v2.patch, 
> 14575-v3.patch, 14575-v4.patch, 14575-v5.patch, 14575-v6.txt, 14575-v6.txt, 
> 14575-v6.txt, 14575.v00.patch
>
>
> Per [~devaraj]'s idea on parent issue, let's see if we can reduce the scope 
> of critical section under which compactions hold the region read lock.
> Here is summary from parent issue:
> Another idea is we can reduce the scope of when the read lock is held during 
> compaction. In theory the compactor only needs a region read lock while 
> deciding what files to compact and at the time of committing the compaction. 
> We're protected from the case of region close events because compactions are 
> checking (every 10k bytes written) if the store has been closed in order to 
> abort in such a case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13471) Fix a possible infinite loop in doMiniBatchMutation

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-13471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011400#comment-15011400
 ] 

Hudson commented on HBASE-13471:


SUCCESS: Integrated in HBase-1.0 #1113 (See 
[https://builds.apache.org/job/HBase-1.0/1113/])
Revert "HBASE-14689 Addendum and unit test for HBASE-13471" (enis: rev 
40ae5bec21c0bb4ce48980e9302db8eb92b40e5d)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java


> Fix a possible infinite loop in doMiniBatchMutation
> ---
>
> Key: HBASE-13471
> URL: https://issues.apache.org/jira/browse/HBASE-13471
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.0.0, 2.0.0, 1.1.0, 0.98.13
>Reporter: Elliott Clark
>Assignee: Rajesh Nishtala
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13471-v1.patch, HBASE-13471.patch
>
>
> {code}
> Thread 4139 
> (regionserver/hbase412.example.com/10.158.6.53:60020-splits-1429003183537):
>   State: WAITING
>   Blocked count: 131
>   Waited count: 228
>   Waiting on 
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@50714dc3
>   Stack:
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> 
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1371)
> org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1325)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:352)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:252)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:509)
> 
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:84)
> 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14575) Reduce scope of compactions holding region lock

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011412#comment-15011412
 ] 

stack commented on HBASE-14575:
---

bq. Browsed some region server log but didn't find exception related to 
compaction.

Sounds good. +1 on commit along with description of logging regime.

> Reduce scope of compactions holding region lock
> ---
>
> Key: HBASE-14575
> URL: https://issues.apache.org/jira/browse/HBASE-14575
> Project: HBase
>  Issue Type: Sub-task
>  Components: Compaction, regionserver
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: 14575-test-case.patch, 14575-v1.patch, 14575-v2.patch, 
> 14575-v3.patch, 14575-v4.patch, 14575-v5.patch, 14575-v6.txt, 14575-v6.txt, 
> 14575-v6.txt, 14575.v00.patch
>
>
> Per [~devaraj]'s idea on parent issue, let's see if we can reduce the scope 
> of critical section under which compactions hold the region read lock.
> Here is summary from parent issue:
> Another idea is we can reduce the scope of when the read lock is held during 
> compaction. In theory the compactor only needs a region read lock while 
> deciding what files to compact and at the time of committing the compaction. 
> We're protected from the case of region close events because compactions are 
> checking (every 10k bytes written) if the store has been closed in order to 
> abort in such a case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14832) Ensure write paths work with ByteBufferedCells in case of compaction

[Anoop Sam John (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011391#comment-15011391
 ] 

Anoop Sam John commented on HBASE-14832:


The write path already have Cell every where and no more ensureKeyValue stuff.. 
 This change will make the write path to work with offheap Cells also so that 
it will help the off heaping of write path as well :-)   +1

> Ensure write paths work with ByteBufferedCells in case of compaction
> 
>
> Key: HBASE-14832
> URL: https://issues.apache.org/jira/browse/HBASE-14832
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver, Scanners
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Fix For: 2.0.0
>
>
> Currently any cell coming out of offheap Bucketcache while compaction does a 
> copy using the getXXXArray() API since write path does not work with BBCells. 
> This JIRA is aimed at changing the write path to support BBCells so that this 
> copy is avoided.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11393) Replication TableCfs should be a PB object rather than a string

[Ashish Singhi (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-11393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011422#comment-15011422
 ] 

Ashish Singhi commented on HBASE-11393:
---

[~chenheng], thanks for the patch.
Did you test this with tableCfs having namespace also included in its table 
name ? I think that is not yet handled, but from high level code review it 
looks like you are almost their.
Can you update the usage in shell script so that user can know how they can 
pass namespace also along with table name.
Can you also add some test where we explicitly set tableCfs having namespace 
also included in its table name.

I see that you are adding tableCfs information also as data to peer id ZK node 
but earlier we had a dedicated ZK node 
{{zookeeper.znode.replication.peers.tableCFs}} for this. So will this be ok ?

Basically what I am looking at is, fixing HBASE-11386 bug, which is very much 
possible here.
I will review more in depth tomorrow and provide my comments.

> Replication TableCfs should be a PB object rather than a string
> ---
>
> Key: HBASE-11393
> URL: https://issues.apache.org/jira/browse/HBASE-11393
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Enis Soztutar
> Fix For: 2.0.0
>
> Attachments: HBASE-11393.patch, HBASE-11393_v1.patch, 
> HBASE-11393_v2.patch, HBASE-11393_v3.patch, HBASE-11393_v4.patch, 
> HBASE-11393_v5.patch, HBASE-11393_v6.patch, HBASE-11393_v7.patch, 
> HBASE-11393_v8.patch, HBASE-11393_v9.patch
>
>
> We concatenate the list of tables and column families in format  
> "table1:cf1,cf2;table2:cfA,cfB" in zookeeper for table-cf to replication peer 
> mapping. 
> This results in ugly parsing code. We should do this a PB object. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14575) Reduce scope of compactions holding region lock

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011398#comment-15011398
 ] 

Ted Yu commented on HBASE-14575:


Browsed some region server log but didn't find exception related to compaction.



> Reduce scope of compactions holding region lock
> ---
>
> Key: HBASE-14575
> URL: https://issues.apache.org/jira/browse/HBASE-14575
> Project: HBase
>  Issue Type: Sub-task
>  Components: Compaction, regionserver
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: 14575-test-case.patch, 14575-v1.patch, 14575-v2.patch, 
> 14575-v3.patch, 14575-v4.patch, 14575-v5.patch, 14575-v6.txt, 14575-v6.txt, 
> 14575-v6.txt, 14575.v00.patch
>
>
> Per [~devaraj]'s idea on parent issue, let's see if we can reduce the scope 
> of critical section under which compactions hold the region read lock.
> Here is summary from parent issue:
> Another idea is we can reduce the scope of when the read lock is held during 
> compaction. In theory the compactor only needs a region read lock while 
> deciding what files to compact and at the time of committing the compaction. 
> We're protected from the case of region close events because compactions are 
> checking (every 10k bytes written) if the store has been closed in order to 
> abort in such a case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14689) Addendum and unit test for HBASE-13471

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011399#comment-15011399
 ] 

Hudson commented on HBASE-14689:


SUCCESS: Integrated in HBase-1.0 #1113 (See 
[https://builds.apache.org/job/HBase-1.0/1113/])
Revert "HBASE-14689 Addendum and unit test for HBASE-13471" (enis: rev 
40ae5bec21c0bb4ce48980e9302db8eb92b40e5d)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java


> Addendum and unit test for HBASE-13471
> --
>
> Key: HBASE-14689
> URL: https://issues.apache.org/jira/browse/HBASE-14689
> Project: HBase
>  Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: hbase-14689_v1-branch-1.1.patch, 
> hbase-14689_v1-branch-1.1.patch, hbase-14689_v1.patch
>
>
> One of our customers ran into HBASE-13471, which resulted in all the handlers 
> getting blocked and various other issues. While backporting the issue, I 
> noticed that there is one more case where we might go into infinite loop. In 
> case a row lock cannot be acquired (due to a previous leak for example which 
> we have seen in Phoenix before) this will cause similar infinite loop. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14815) TestMobExportSnapshot.testExportFailure timeout occasionally

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-14815:
--
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Excellent. Thanks for digging in [~chenheng] It just started to come up on my 
radar. Committed. Thanks.

> TestMobExportSnapshot.testExportFailure timeout occasionally
> 
>
> Key: HBASE-14815
> URL: https://issues.apache.org/jira/browse/HBASE-14815
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Heng Chen
>Assignee: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-14815.patch
>
>
> On master,  TestMobExportSnapshot.testExportFailure timeout occasionally.
> See
> https://builds.apache.org/job/PreCommit-HBASE-Build/16514//testReport/org.apache.hadoop.hbase.snapshot/TestMobExportSnapshot/testExportFailure/
> https://builds.apache.org/job/PreCommit-HBASE-Build/16511//testReport/org.apache.hadoop.hbase.snapshot/TestMobExportSnapshot/testExportFailure/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14575) Reduce scope of compactions holding region lock

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011418#comment-15011418
 ] 

Ted Yu commented on HBASE-14575:


bq. with description of logging regime

You meant 'locking regime', I guess.

> Reduce scope of compactions holding region lock
> ---
>
> Key: HBASE-14575
> URL: https://issues.apache.org/jira/browse/HBASE-14575
> Project: HBase
>  Issue Type: Sub-task
>  Components: Compaction, regionserver
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: 14575-test-case.patch, 14575-v1.patch, 14575-v2.patch, 
> 14575-v3.patch, 14575-v4.patch, 14575-v5.patch, 14575-v6.txt, 14575-v6.txt, 
> 14575-v6.txt, 14575.v00.patch
>
>
> Per [~devaraj]'s idea on parent issue, let's see if we can reduce the scope 
> of critical section under which compactions hold the region read lock.
> Here is summary from parent issue:
> Another idea is we can reduce the scope of when the read lock is held during 
> compaction. In theory the compactor only needs a region read lock while 
> deciding what files to compact and at the time of committing the compaction. 
> We're protected from the case of region close events because compactions are 
> checking (every 10k bytes written) if the store has been closed in order to 
> abort in such a case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow

[Jesse Yates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011480#comment-15011480
 ] 

Jesse Yates commented on HBASE-14703:
-

bq. remove some code unused in RpcRetryingCallerImpl

I left it, to preserve the current implementation. I don't know where else it 
would be used, but can't hurt to not change things.

Then here, in AsyncProcess
{code}
723   // As mutateRow, if statistic off, nothing will 
return. So let's stop.
724   actionsInProgress.set(0);
{code}

This doesn't feel right. In receiveMultiAction -> setResult we just decrement 
the actionsInProgress, rather than set it to 0. We don't have any tests that 
check running ops concurrently, so I think it would be better to refactor 
AsyncProcess a little bit more to unify the logic a bit

> not collect stats when call HTable.mutateRow 
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
>  Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, 
> HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, HBASE-14703.patch, 
> HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, 
> HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update 
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}}  by 
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we 
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
>   @Override
>   public T callWithRetries(RetryingCallable callable, int callTimeout)
>   throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
>   }
>   @Override
>   public T callWithoutRetries(RetryingCallable callable, int callTimeout)
>   throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
>   }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do 
> update again. 
> {code}
> // update the stats about the region, if its a user table. We don't want to 
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
>   result = ResultStatsUtil.updateStats(result,
>   AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary,  remove it?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14221) Reduce the number of time row comparison is done in a Scan

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011495#comment-15011495
 ] 

stack commented on HBASE-14221:
---

Did you mean an issue other than HBASE-14221 above [~ram_krish]? 14221 is this 
issue.

Is there a write up or class diagram of how our merge sort works anywhere? Even 
a listing of participants would help (filters, fake keys).

>From your description, it sounds like the patch here is at a different level. 
>Let me take a look.

Thanks for digging in here [~ram_krish] I think there are some potential big 
wins here but takes doing a deep dig. The original code is from another time.




> Reduce the number of time row comparison is done in a Scan
> --
>
> Key: HBASE-14221
> URL: https://issues.apache.org/jira/browse/HBASE-14221
> Project: HBase
>  Issue Type: Sub-task
>  Components: Scanners
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> Attachments: 14221-0.98-takeALook.txt, HBASE-14221.patch, 
> HBASE-14221_1.patch, HBASE-14221_1.patch, HBASE-14221_6.patch, 
> withmatchingRowspatch.png, withoutmatchingRowspatch.png
>
>
> When we tried to do some profiling with the PE tool found this.
> Currently we do row comparisons in 3 places in a simple Scan case.
> 1) ScanQueryMatcher
> {code}
>int ret = this.rowComparator.compareRows(curCell, cell);
> if (!this.isReversed) {
>   if (ret <= -1) {
> return MatchCode.DONE;
>   } else if (ret >= 1) {
> // could optimize this, if necessary?
> // Could also be called SEEK_TO_CURRENT_ROW, but this
> // should be rare/never happens.
> return MatchCode.SEEK_NEXT_ROW;
>   }
> } else {
>   if (ret <= -1) {
> return MatchCode.SEEK_NEXT_ROW;
>   } else if (ret >= 1) {
> return MatchCode.DONE;
>   }
> }
> {code}
> 2) In StoreScanner next() while starting to scan the row
> {code}
> if (!scannerContext.hasAnyLimit(LimitScope.BETWEEN_CELLS) || 
> matcher.curCell == null ||
> isNewRow || !CellUtil.matchingRow(peeked, matcher.curCell)) {
>   this.countPerRow = 0;
>   matcher.setToNewRow(peeked);
> }
> {code}
> Particularly to see if we are in a new row.
> 3) In HRegion
> {code}
>   scannerContext.setKeepProgress(true);
>   heap.next(results, scannerContext);
>   scannerContext.setKeepProgress(tmpKeepProgress);
>   nextKv = heap.peek();
> moreCellsInRow = moreCellsInRow(nextKv, currentRowCell);
> {code}
> Here again there are cases where we need to careful for a MultiCF case.  Was 
> trying to solve this for the MultiCF case but is having lot of cases to 
> solve. But atleast for a single CF case I think these comparison can be 
> reduced.
> So for a single CF case in the SQM we are able to find if we have crossed a 
> row using the code pasted above in SQM. That comparison is definitely needed.
> Now in case of a single CF the HRegion is going to have only one element in 
> the heap and so the 3rd comparison can surely be avoided if the 
> StoreScanner.next() was over due to MatchCode.DONE caused by SQM.
> Coming to the 2nd compareRows that we do in StoreScanner. next() - even that 
> can be avoided if we know that the previous next() call was over due to a new 
> row. Doing all this I found that the compareRows in the profiler which was 
> 19% got reduced to 13%. Initially we can solve for single CF case which can 
> be extended to MultiCF cases.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14329) Report region in transition only ever operates on one region

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-14329:

Issue Type: Bug  (was: Sub-task)
Parent: (was: HBASE-14238)

> Report region in transition only ever operates on one region
> 
>
> Key: HBASE-14329
> URL: https://issues.apache.org/jira/browse/HBASE-14329
> Project: HBase
>  Issue Type: Bug
>  Components: Region Assignment
>Reporter: Elliott Clark
> Fix For: 1.2.0
>
>
> Report region in transition takes a list of regions but it only ever operates 
> on one region however more than one region can be reported.
> Seems like this could cause some serious weirdness on failure cases.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14829) Add more checkstyles

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011334#comment-15011334
 ] 

stack commented on HBASE-14829:
---

The site build failed with this [~appy]

{code}
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on project 
hbase: Error generating maven-checkstyle-plugin:2.16:checkstyle: Failed during 
checkstyle configuration: cannot initialize module TreeWalker - Property 
'sortStaticImportsAlphabetically' in module ImportOrder does not exist, please 
check the documentation -> [Help 1]
[ERROR] 
{code}



> Add more checkstyles
> 
>
> Key: HBASE-14829
> URL: https://issues.apache.org/jira/browse/HBASE-14829
> Project: HBase
>  Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-14829-master.patch
>
>
> This jira will add following checkstyles:
> [ImportOrder|http://checkstyle.sourceforge.net/config_imports.html#ImportOrder]
>  : keep imports in sorted order
> [LeftCurly|http://checkstyle.sourceforge.net/config_blocks.html#LeftCurly] : 
> Placement of left curly brace. Does 'eol' sounds right setting?
> [NeedBraces|http://checkstyle.sourceforge.net/config_blocks.html#NeedBraces] 
> : braces around code blocks
> [JavadocTagContinuationIndentation|http://checkstyle.sourceforge.net/config_javadoc.html#JavadocTagContinuationIndentation]
>  : Avoid weird indentations in javadocs
> [NonEmptyAtclauseDescription|http://checkstyle.sourceforge.net/config_javadoc.html#NonEmptyAtclauseDescription]
>  : We have so many empty javadoc @ clauses. This'll take care of it.
>  
> [Indentation|http://checkstyle.sourceforge.net/config_misc.html#Indentation] 
> : Bad indentation hurts code readability. We have indentation guidelines, 
> should be fine enforcing them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-14238) Branch-1.2 AM issues

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-14238.
-
Resolution: Fixed

all subtasks are complete, resolving.

> Branch-1.2 AM issues
> 
>
> Key: HBASE-14238
> URL: https://issues.apache.org/jira/browse/HBASE-14238
> Project: HBase
>  Issue Type: Bug
>  Components: Region Assignment
>Affects Versions: 1.2.0
>Reporter: Elliott Clark
>Priority: Blocker
> Fix For: 1.2.0
>
>
> Parent jira for issues found through IT tests with Chaos monkey.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14329) Report region in transition only ever operates on one region

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-14329:

Fix Version/s: (was: 1.2.0)
   1.3.0

> Report region in transition only ever operates on one region
> 
>
> Key: HBASE-14329
> URL: https://issues.apache.org/jira/browse/HBASE-14329
> Project: HBase
>  Issue Type: Bug
>  Components: Region Assignment
>Reporter: Elliott Clark
> Fix For: 1.3.0
>
>
> Report region in transition takes a list of regions but it only ever operates 
> on one region however more than one region can be reported.
> Seems like this could cause some serious weirdness on failure cases.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14829) Add more checkstyles

[Jonathan Hsieh (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011326#comment-15011326
 ] 

Jonathan Hsieh commented on HBASE-14829:


Hm.. looks like the post patch checkstyle run may not have run 

https://builds.apache.org/job/PreCommit-HBASE-Build/16563/artifact/patchprocess/


> Add more checkstyles
> 
>
> Key: HBASE-14829
> URL: https://issues.apache.org/jira/browse/HBASE-14829
> Project: HBase
>  Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-14829-master.patch
>
>
> This jira will add following checkstyles:
> [ImportOrder|http://checkstyle.sourceforge.net/config_imports.html#ImportOrder]
>  : keep imports in sorted order
> [LeftCurly|http://checkstyle.sourceforge.net/config_blocks.html#LeftCurly] : 
> Placement of left curly brace. Does 'eol' sounds right setting?
> [NeedBraces|http://checkstyle.sourceforge.net/config_blocks.html#NeedBraces] 
> : braces around code blocks
> [JavadocTagContinuationIndentation|http://checkstyle.sourceforge.net/config_javadoc.html#JavadocTagContinuationIndentation]
>  : Avoid weird indentations in javadocs
> [NonEmptyAtclauseDescription|http://checkstyle.sourceforge.net/config_javadoc.html#NonEmptyAtclauseDescription]
>  : We have so many empty javadoc @ clauses. This'll take care of it.
>  
> [Indentation|http://checkstyle.sourceforge.net/config_misc.html#Indentation] 
> : Bad indentation hurts code readability. We have indentation guidelines, 
> should be fine enforcing them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14703) not collect stats when call HTable.mutateRow

[Heng Chen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Heng Chen updated HBASE-14703:
--
Attachment: HBASE-14703_v5.2.patch

> not collect stats when call HTable.mutateRow 
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
>  Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, 
> HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, HBASE-14703.patch, 
> HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, 
> HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update 
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}}  by 
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we 
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
>   @Override
>   public T callWithRetries(RetryingCallable callable, int callTimeout)
>   throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
>   }
>   @Override
>   public T callWithoutRetries(RetryingCallable callable, int callTimeout)
>   throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
>   }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do 
> update again. 
> {code}
> // update the stats about the region, if its a user table. We don't want to 
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
>   result = ResultStatsUtil.updateStats(result,
>   AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary,  remove it?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow

[Heng Chen (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011221#comment-15011221
 ] 

Heng Chen commented on HBASE-14703:
---

Update patch to fix failed testcase.
It is here in AsyncProcess.run,  
{code}
+  if (currentCallable != null) {
+if (res == null|| res.getResults().size() == 0) {
+  // As mutateRow, if statistic off, nothing will return. So 
let's stop.
+  actionsInProgress.set(0);
+  return;
+}
+  } else {
+if (res == null) {
+  return;
+}
+  } 
{code}
As for put, it will has a response, and res.getResults().size() is 0,  we 
should collect exceptions in receiveMultiAction(multiAction, server, res, 
numAttempt),  not just return.

Let's see if we could unify this code for mutateRow and put,

> not collect stats when call HTable.mutateRow 
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
>  Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, 
> HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, HBASE-14703.patch, 
> HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, 
> HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update 
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}}  by 
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we 
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
>   @Override
>   public T callWithRetries(RetryingCallable callable, int callTimeout)
>   throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
>   }
>   @Override
>   public T callWithoutRetries(RetryingCallable callable, int callTimeout)
>   throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
>   }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do 
> update again. 
> {code}
> // update the stats about the region, if its a user table. We don't want to 
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
>   result = ResultStatsUtil.updateStats(result,
>   AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary,  remove it?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14832) Ensure write paths work with ByteBufferedCells in case of compaction

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011228#comment-15011228
 ] 

stack commented on HBASE-14832:
---

+1

> Ensure write paths work with ByteBufferedCells in case of compaction
> 
>
> Key: HBASE-14832
> URL: https://issues.apache.org/jira/browse/HBASE-14832
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver, Scanners
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Fix For: 2.0.0
>
>
> Currently any cell coming out of offheap Bucketcache while compaction does a 
> copy using the getXXXArray() API since write path does not work with BBCells. 
> This JIRA is aimed at changing the write path to support BBCells so that this 
> copy is avoided.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14828) Deprecate stateful REST scanners

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011322#comment-15011322
 ] 

stack commented on HBASE-14828:
---

bq. Stateful scanners are by definition not REST-ful.

lol

bq. Deprecate in 1.x and remove in 2.x? Too sudden?

I'd say so, this late in the 1.x game. Deprecate in 1.1 and remove in 3.0?

> Deprecate stateful REST scanners
> 
>
> Key: HBASE-14828
> URL: https://issues.apache.org/jira/browse/HBASE-14828
> Project: HBase
>  Issue Type: Task
>Reporter: Andrew Purtell
>
> Deprecate the REST gateway's stateful scanner API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14329) Report region in transition only ever operates on one region

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011151#comment-15011151
 ] 

Sean Busbey commented on HBASE-14329:
-

converted from subtask to issue and moved out of 1.2

> Report region in transition only ever operates on one region
> 
>
> Key: HBASE-14329
> URL: https://issues.apache.org/jira/browse/HBASE-14329
> Project: HBase
>  Issue Type: Bug
>  Components: Region Assignment
>Reporter: Elliott Clark
> Fix For: 1.3.0
>
>
> Report region in transition takes a list of regions but it only ever operates 
> on one region however more than one region can be reported.
> Seems like this could cause some serious weirdness on failure cases.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14238) Branch-1.2 AM issues

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-14238:

Assignee: Elliott Clark

> Branch-1.2 AM issues
> 
>
> Key: HBASE-14238
> URL: https://issues.apache.org/jira/browse/HBASE-14238
> Project: HBase
>  Issue Type: Bug
>  Components: Region Assignment
>Affects Versions: 1.2.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Blocker
> Fix For: 1.2.0
>
>
> Parent jira for issues found through IT tests with Chaos monkey.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011298#comment-15011298
 ] 

stack commented on HBASE-14498:
---

[~pankaj_kumar] Thanks for the response.

I think I understand how the failure happens from your original description. My 
question was how the test replicates the failure seen. There are not masters 
involved in the added test.

bq. The time interval (t) should be less than the ZK Session time out. (May be 
2/3rd of session time out value ) , This is to make sure that standby HM will 
not become active within this time period.

Understood. Was afraid the timeout we ask for might be different from what zk 
sets on the session and then this math would be off. What you have here should 
be fine then. I need to dig up more on when zk gives us a session that is other 
than what we asked for. Onus is on me.

bq. >> Yeah daemon thread will be spawned and will be active util 
connWaitTimeOut or SyncConnected.

Could we spawn many of these threads? Should we ever spawn more than one?

Thanks.

> Master stuck in infinite loop when all Zookeeper servers are unreachable
> 
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
>  Issue Type: Bug
>  Components: master
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Blocker
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4
>
> Attachments: HBASE-14498-V2.patch, HBASE-14498-V3.patch, 
> HBASE-14498-V4.patch, HBASE-14498.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to 
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO 
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)] 
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in 
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and 
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO 
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)] 
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)] 
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN 
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)] 
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO 
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)] 
> zookeeper.ClientCnxn: Opening socket connection to server 
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown 
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)] 
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)] 
> zookeeper.ClientCnxn: Opening socket connection to server 
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown 
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)] 
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in 
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and 
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000] 
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, 
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181, 
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException: 
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)] 
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)] 
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)] 
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181. 
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt 
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an 
> > active master. 
> > HM2 is keep on waiting for region server to report him as part of active 
> > master intialization.
> {noformat} 
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting 
> for region servers count to settle; currently checked in 0, slept for 0 ms, 
> expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval 
> of 1500 ms. | 
> 

[jira] [Resolved] (HBASE-10588) Remove deprecated public facing classes/methods related to KeyValue

[Jonathan Hsieh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh resolved HBASE-10588.

Resolution: Won't Fix

closing, other issues have been filed.

> Remove deprecated public facing classes/methods related to KeyValue
> ---
>
> Key: HBASE-10588
> URL: https://issues.apache.org/jira/browse/HBASE-10588
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Jimmy Xiang
>Assignee: Rekha Joshi
> Fix For: 2.0.0
>
>
> Due to backward compatibility issue, old public facing classes/methods 
> related to KeyValue are deprecated.  We should clean them up after they have 
> been deprecated for a while.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14822) Renewing leases of scanners doesn't work

[Samarth Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Samarth Jain updated HBASE-14822:
-
Attachment: 14822-0.98-v4.txt

Patch with checkstyle errors fixed. The diff looks more complex than it really 
is because of indentation changes. I have added the below check on top of Lar's 
patch in HRegionServer.java :

{code}
+  if (rows > 0) {
 // Limit the initial allocation of the result array to the minimum
 // of 'rows' or 100
{code}




> Renewing leases of scanners doesn't work
> 
>
> Key: HBASE-14822
> URL: https://issues.apache.org/jira/browse/HBASE-14822
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.98.14
>Reporter: Samarth Jain
>Assignee: Lars Hofhansl
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17, 1.0.4
>
> Attachments: 14822-0.98-v2.txt, 14822-0.98-v3.txt, 14822-0.98-v4.txt, 
> 14822-0.98.txt
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14575) Reduce scope of compactions holding region lock

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-14575:
---
Attachment: 14575-v7.txt

Patch v7 adds comment to HRegion#compact(), explaining the rationale behind the 
relaxation of region lock.

It also increases the frequency of compaction requests in 
TestHRegionServerBulkLoad

> Reduce scope of compactions holding region lock
> ---
>
> Key: HBASE-14575
> URL: https://issues.apache.org/jira/browse/HBASE-14575
> Project: HBase
>  Issue Type: Sub-task
>  Components: Compaction, regionserver
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: 14575-test-case.patch, 14575-v1.patch, 14575-v2.patch, 
> 14575-v3.patch, 14575-v4.patch, 14575-v5.patch, 14575-v6.txt, 14575-v6.txt, 
> 14575-v6.txt, 14575-v7.txt, 14575.v00.patch
>
>
> Per [~devaraj]'s idea on parent issue, let's see if we can reduce the scope 
> of critical section under which compactions hold the region read lock.
> Here is summary from parent issue:
> Another idea is we can reduce the scope of when the read lock is held during 
> compaction. In theory the compactor only needs a region read lock while 
> deciding what files to compact and at the time of committing the compaction. 
> We're protected from the case of region close events because compactions are 
> checking (every 10k bytes written) if the store has been closed in order to 
> abort in such a case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-9245) Remove dead or deprecated code from hbase 0.96

[Jonathan Hsieh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-9245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh resolved HBASE-9245.
---
Resolution: Not A Problem

0.96 has long past.  closing this out.

> Remove dead or deprecated code from hbase 0.96
> --
>
> Key: HBASE-9245
> URL: https://issues.apache.org/jira/browse/HBASE-9245
> Project: HBase
>  Issue Type: Bug
>Reporter: Jonathan Hsieh
>
> This is an umbrella issue that will cover the removal or refactoring of 
> dangling dead code and cruft.  Some can make it into 0.96, some may have to 
> wait for an 0.98.  The "great culling" of code will be grouped patches that 
> are logically related.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14575) Reduce scope of compactions holding region lock

[Devaraj Das (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011560#comment-15011560
 ] 

Devaraj Das commented on HBASE-14575:
-

Hmm.. On the unit test, I am not sure it's going to be very effective unless we 
make sure compaction is underway when a multi-cf bulkload is attempted. 
[~yuzhih...@gmail.com] can you please change the test to do that.

> Reduce scope of compactions holding region lock
> ---
>
> Key: HBASE-14575
> URL: https://issues.apache.org/jira/browse/HBASE-14575
> Project: HBase
>  Issue Type: Sub-task
>  Components: Compaction, regionserver
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: 14575-test-case.patch, 14575-v1.patch, 14575-v2.patch, 
> 14575-v3.patch, 14575-v4.patch, 14575-v5.patch, 14575-v6.txt, 14575-v6.txt, 
> 14575-v6.txt, 14575.v00.patch
>
>
> Per [~devaraj]'s idea on parent issue, let's see if we can reduce the scope 
> of critical section under which compactions hold the region read lock.
> Here is summary from parent issue:
> Another idea is we can reduce the scope of when the read lock is held during 
> compaction. In theory the compactor only needs a region read lock while 
> deciding what files to compact and at the time of committing the compaction. 
> We're protected from the case of region close events because compactions are 
> checking (every 10k bytes written) if the store has been closed in order to 
> abort in such a case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-9405) Remove shim stuff from Filter

[Jonathan Hsieh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-9405:
--
Issue Type: Bug  (was: Sub-task)
Parent: (was: HBASE-9245)

> Remove shim stuff from Filter
> -
>
> Key: HBASE-9405
> URL: https://issues.apache.org/jira/browse/HBASE-9405
> Project: HBase
>  Issue Type: Bug
>Reporter: Jonathan Hsieh
>
> Aleks's compat report shows that Filter was an interface in 0.94 and is now 
> an abstract class in 0.95/0.96.  That means the shims aren't useful at all.  
> Might as well remove them and add release notes requiring 3rd party filters 
> to be modified to work on 0.96.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14818) user_permission does not list namespace permissions

[Jerry He (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011661#comment-15011661
 ] 

Jerry He commented on HBASE-14818:
--

bq. the command with '.*' option to list all permissions
It makes sense.

bq. Alternatively I can think of a command called group_permission that will 
list all group permissions.
The user_permission command lists the group permissions as well, right?

> user_permission does not list namespace permissions
> ---
>
> Key: HBASE-14818
> URL: https://issues.apache.org/jira/browse/HBASE-14818
> Project: HBase
>  Issue Type: Bug
>  Components: hbase
>Affects Versions: 1.0.2
>Reporter: Steven Hancz
>Priority: Minor
>
> The user_permission command does not list namespace permissions:
> For example: if I create a new namespace or use an existing namespace and 
> grant a user privileges to that namespace the command user_permission does 
> not list the. The permission is visible in the acl table.
> Example:
> hbase(main):005:0>  create_namespace 'ns3'
> 0 row(s) in 0.1640 seconds
> hbase(main):007:0> grant 'test_user','RWXAC','@ns3'
> 0 row(s) in 0.5680 seconds
> hbase(main):008:0> user_permission '.*'
> User   
> Namespace,Table,Family,Qualifier:Permission   
>  
>  sh82993   finance,finance:emp,,: [Permission: 
> actions=READ,WRITE,EXEC,CREATE,ADMIN]  
>  @hbaseglobaldba   hbase,hbase:acl,,: [Permission: 
> actions=EXEC,CREATE,ADMIN] 
>  @hbaseglobaloper  hbase,hbase:acl,,: [Permission: 
> actions=EXEC,ADMIN]
>  hdfs  hbase,hbase:acl,,: [Permission: 
> actions=READ,WRITE,CREATE,ADMIN,EXEC]  
>  sh82993   ns1,ns1:tbl1,,: [Permission: 
> actions=READ,WRITE,EXEC,CREATE,ADMIN] 
>  ns1admin  ns1,ns1:tbl2,,: [Permission: 
> actions=EXEC,CREATE,ADMIN]
>  @hbaseappltest_ns1funct   ns1,ns1:tbl2,,: [Permission: 
> actions=READ,WRITE,EXEC]  
>  ns1funct  ns1,ns1:tbl2,,: [Permission: 
> actions=READ,WRITE,EXEC,CREATE,ADMIN] 
>  hbase ns2,ns2:tbl1,,: [Permission: 
> actions=READ,WRITE,EXEC,CREATE,ADMIN] 
> 9 row(s) in 1.8090 seconds
> As you can see user test_user does not appear in the output, but on can see 
> the permission in the ACL table. 
> hbase(main):001:0>  scan 'hbase:acl'
> ROWCOLUMN+CELL
> 
>  @finance  column=l:sh82993, timestamp=105519510, 
> value=RWXCA 
>  @gcbcppdn column=l:hdfs, timestamp=1446141119602, 
> value=RWCXA
>  @hbasecolumn=l:hdfs, timestamp=1446141485136, 
> value=RWCAX
>  @ns1  column=l:@hbaseappltest_ns1admin, 
> timestamp=1447437007467, value=RWXCA 
>  @ns1  column=l:@hbaseappltest_ns1funct, 
> timestamp=1447427366835, value=RWX   
>  @ns2  column=l:@hbaseappltest_ns2admin, 
> timestamp=1446674470456, value=XCA   
>  @ns2  column=l:test_user, 
> timestamp=1447692840030, value=RWAC   
>  
>  @ns3  column=l:test_user, 
> timestamp=1447692860434, value=RWXAC  
>  
>  finance:emp   column=l:sh82993, timestamp=107723316, 
> value=RWXCA 
>  hbase:acl column=l:@hbaseglobaldba, 
> timestamp=1446590375370, value=XCA   
>  hbase:acl column=l:@hbaseglobaloper, 
> timestamp=1446590387965, value=XA   
>  hbase:acl column=l:hdfs, timestamp=1446141737213, 
> value=RWCAX
>  ns1:tbl1  column=l:sh82993, timestamp=1446674153058, 
> value=RWXCA 
>  ns1:tbl2  column=l:@hbaseappltest_ns1funct, 

[jira] [Updated] (HBASE-14575) Relax region read lock for compactions

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-14575:
---
Summary: Relax region read lock for compactions  (was: Reduce scope of 
compactions holding region lock)

> Relax region read lock for compactions
> --
>
> Key: HBASE-14575
> URL: https://issues.apache.org/jira/browse/HBASE-14575
> Project: HBase
>  Issue Type: Sub-task
>  Components: Compaction, regionserver
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: 14575-test-case.patch, 14575-v1.patch, 14575-v2.patch, 
> 14575-v3.patch, 14575-v4.patch, 14575-v5.patch, 14575-v6.txt, 14575-v6.txt, 
> 14575-v6.txt, 14575-v7.txt, 14575.v00.patch
>
>
> Per [~devaraj]'s idea on parent issue, let's see if we can reduce the scope 
> of critical section under which compactions hold the region read lock.
> Here is summary from parent issue:
> Another idea is we can reduce the scope of when the read lock is held during 
> compaction. In theory the compactor only needs a region read lock while 
> deciding what files to compact and at the time of committing the compaction. 
> We're protected from the case of region close events because compactions are 
> checking (every 10k bytes written) if the store has been closed in order to 
> abort in such a case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-9396) Remove deprecation and shims from HBASE-9334/HBASE-9359 from 0.98

[Jonathan Hsieh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-9396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh resolved HBASE-9396.
---
Resolution: Won't Fix

ancient.  closing out.

> Remove deprecation and shims from HBASE-9334/HBASE-9359 from 0.98
> -
>
> Key: HBASE-9396
> URL: https://issues.apache.org/jira/browse/HBASE-9396
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
>
> The version commited to 0.96 and trunk were essentially identical and 
> included deprecation shims.  This follow-on patch will remove deprecated 
> stuff from trunk only.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-9381) Check that RawComparator implementations use faster UnsafeComparer

[Jonathan Hsieh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-9381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh resolved HBASE-9381.
---
Resolution: Won't Fix

ancient.  closing this out.

> Check that RawComparator implementations use faster UnsafeComparer
> --
>
> Key: HBASE-9381
> URL: https://issues.apache.org/jira/browse/HBASE-9381
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Jonathan Hsieh
>
> Follow on from review of HBASE-9247.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14777) Replication fails with IndexOutOfBoundsException

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-14777:
---
Hadoop Flags: Reviewed
  Status: Patch Available  (was: Open)

+1 if tests pass.

> Replication fails with IndexOutOfBoundsException
> 
>
> Key: HBASE-14777
> URL: https://issues.apache.org/jira/browse/HBASE-14777
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Reporter: Bhupendra Kumar Jain
>Assignee: Ashu Pachauri
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14777-1.patch, HBASE-14777-2.patch, 
> HBASE-14777-3.patch, HBASE-14777.patch
>
>
> Replication fails with IndexOutOfBoundsException 
> {code}
> regionserver.ReplicationSource$ReplicationSourceWorkerThread(939): 
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint
>  threw unknown exception:java.lang.IndexOutOfBoundsException: Index: 1, Size: 
> 1
>   at java.util.ArrayList.rangeCheck(Unknown Source)
>   at java.util.ArrayList.remove(Unknown Source)
>   at 
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicate(HBaseInterClusterReplicationEndpoint.java:222)
> {code}
> Its happening due to incorrect removal of entries from the replication 
> entries list. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-14833) HFileOutputFormat2 should allow for custom reducer logic

[Gowtam Lal (JIRA)]

Gowtam Lal created HBASE-14833:
--

 Summary: HFileOutputFormat2 should allow for custom reducer logic
 Key: HBASE-14833
 URL: https://issues.apache.org/jira/browse/HBASE-14833
 Project: HBase
  Issue Type: Improvement
  Components: API, mapreduce
Reporter: Gowtam Lal
Priority: Minor


Right now, HFileOutputFormat2.configureIncrementalLoad() will configure a 
ReducerClass which takes all input and passes it through to be written to an 
HFile. Unfortunately, this prevents a user from plugging in his or her own 
logic for the Reduce phase. 

TableMapReduceUtil.initTableReducerJob() accepts a custom ReducerClass, 
allowing users to operate on the tuples coming out of the MapperClass before 
they're written to output. Could HFileOutputFormat2.configureIncrementalLoad() 
possibly do something similar?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011726#comment-15011726
 ] 

Hadoop QA commented on HBASE-14703:
---

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12773013/HBASE-14703_v5.2.patch
  against master branch at commit 5ebd7660a94bfb18e6e05b6e46195c76c099eda2.
  ATTACHMENT ID: 12773013

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 16 new 
or modified tests.

{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0 
2.7.1)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 protoc{color}.  The applied patch does not increase the 
total number of protoc compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any  new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16571//testReport/
Release Findbugs (version 2.0.3)warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16571//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16571//artifact/patchprocess/checkstyle-aggregate.html

  Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16571//console

This message is automatically generated.

> not collect stats when call HTable.mutateRow 
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
>  Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, 
> HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, HBASE-14703.patch, 
> HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, 
> HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update 
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}}  by 
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we 
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
>   @Override
>   public T callWithRetries(RetryingCallable callable, int callTimeout)
>   throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
>   }
>   @Override
>   public T callWithoutRetries(RetryingCallable callable, int callTimeout)
>   throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
>   }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do 
> update again. 
> {code}
> // update the stats about the region, if its a user table. We don't want to 
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
>   result = ResultStatsUtil.updateStats(result,
>   AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary,  remove it?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14782) FuzzyRowFilter skips valid rows

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011743#comment-15011743
 ] 

Vladimir Rodionov commented on HBASE-14782:
---

{quote}
 but if other guys want to call this method in some other places? They don't 
know we should ensure result.length>=fuzzyKeyMeta.length
{quote}

Trying to figure out on why anyone will ever want to re-use the method which is 
specific to a particular algorithm? This is not a public API.

> FuzzyRowFilter skips valid rows
> ---
>
> Key: HBASE-14782
> URL: https://issues.apache.org/jira/browse/HBASE-14782
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4, 0.98.17
>
> Attachments: HBASE-14782-0.98-v4.patch, HBASE-14782-v3.patch, 
> HBASE-14782-v4.patch, HBASE-14782.patch, HBASE-14782.patch
>
>
> The issue may affect not only master branch, but previous releases as well.
> This is from one of our customers:
> {quote}
> We are experiencing a problem with the FuzzyRowFilter for HBase scan. We 
> think that it is a bug. 
> Fuzzy filter should pick a row if it matches filter criteria irrespective of 
> other rows present in table but filter is dropping a row depending on some 
> other row present in table. 
> Details/Step to reproduce/Sample outputs below: 
> Missing row key: \x9C\x00\x044\x00\x00\x00\x00 
> Causing row key: \x9C\x00\x03\xE9e\xBB{X\x1Fwts\x1F\x15vRX 
> Prerequisites 
> 1. Create a test table. HBase shell command -- create 'fuzzytest','d' 
> 2. Insert some test data. HBase shell commands: 
> • put 'fuzzytest',"\x9C\x00\x044\x00\x00\x00\x00",'d:a','junk' 
> • put 'fuzzytest',"\x9C\x00\x044\x01\x00\x00\x00",'d:a','junk' 
> • put 'fuzzytest',"\x9C\x00\x044\x00\x01\x00\x00",'d:a','junk' 
> • put 'fuzzytest',"\x9C\x00\x044\x00\x00\x01\x00",'d:a','junk' 
> • put 'fuzzytest',"\x9C\x00\x044\x00\x01\x00\x01",'d:a','junk' 
> • put 'fuzzytest',"\x9B\x00\x044e\xBB\xB2\xBB",'d:a','junk' 
> • put 'fuzzytest',"\x9D\x00\x044e\xBB\xB2\xBB",'d:a','junk' 
> Now when you run the code, you will find \x9C\x00\x044\x00\x00\x00\x00 in 
> output because it matches filter criteria. (Refer how to run code below) 
> Insert the row key causing bug: 
> HBase shell command: put 
> 'fuzzytest',"\x9C\x00\x03\xE9e\xBB{X\x1Fwts\x1F\x15vRX",'d:a','junk' 
> Now when you run the code, you will not find \x9C\x00\x044\x00\x00\x00\x00 in 
> output even though it still matches filter criteria. 
> {quote}
> Verified the issue on master.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14815) TestMobExportSnapshot.testExportFailure timeout occasionally

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011730#comment-15011730
 ] 

Hudson commented on HBASE-14815:


FAILURE: Integrated in HBase-Trunk_matrix #478 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/478/])
HBASE-14815 TestMobExportSnapshot.testExportFailure timeout occasionally 
(stack: rev b2187c31ab8294215aab46db5dcd3b0dce4fad65)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java


> TestMobExportSnapshot.testExportFailure timeout occasionally
> 
>
> Key: HBASE-14815
> URL: https://issues.apache.org/jira/browse/HBASE-14815
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Heng Chen
>Assignee: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-14815.patch
>
>
> On master,  TestMobExportSnapshot.testExportFailure timeout occasionally.
> See
> https://builds.apache.org/job/PreCommit-HBASE-Build/16514//testReport/org.apache.hadoop.hbase.snapshot/TestMobExportSnapshot/testExportFailure/
> https://builds.apache.org/job/PreCommit-HBASE-Build/16511//testReport/org.apache.hadoop.hbase.snapshot/TestMobExportSnapshot/testExportFailure/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14818) user_permission does not list namespace permissions

[Steven Hancz (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011852#comment-15011852
 ] 

Steven Hancz commented on HBASE-14818:
--

Hi Jerry,

user_permission does display group permissions. Hopefully it will be picked up 
by someone and a patch will be developed to show namespace permissions as well.

Steven

> user_permission does not list namespace permissions
> ---
>
> Key: HBASE-14818
> URL: https://issues.apache.org/jira/browse/HBASE-14818
> Project: HBase
>  Issue Type: Bug
>  Components: hbase
>Affects Versions: 1.0.2
>Reporter: Steven Hancz
>Priority: Minor
>
> The user_permission command does not list namespace permissions:
> For example: if I create a new namespace or use an existing namespace and 
> grant a user privileges to that namespace the command user_permission does 
> not list the. The permission is visible in the acl table.
> Example:
> hbase(main):005:0>  create_namespace 'ns3'
> 0 row(s) in 0.1640 seconds
> hbase(main):007:0> grant 'test_user','RWXAC','@ns3'
> 0 row(s) in 0.5680 seconds
> hbase(main):008:0> user_permission '.*'
> User   
> Namespace,Table,Family,Qualifier:Permission   
>  
>  sh82993   finance,finance:emp,,: [Permission: 
> actions=READ,WRITE,EXEC,CREATE,ADMIN]  
>  @hbaseglobaldba   hbase,hbase:acl,,: [Permission: 
> actions=EXEC,CREATE,ADMIN] 
>  @hbaseglobaloper  hbase,hbase:acl,,: [Permission: 
> actions=EXEC,ADMIN]
>  hdfs  hbase,hbase:acl,,: [Permission: 
> actions=READ,WRITE,CREATE,ADMIN,EXEC]  
>  sh82993   ns1,ns1:tbl1,,: [Permission: 
> actions=READ,WRITE,EXEC,CREATE,ADMIN] 
>  ns1admin  ns1,ns1:tbl2,,: [Permission: 
> actions=EXEC,CREATE,ADMIN]
>  @hbaseappltest_ns1funct   ns1,ns1:tbl2,,: [Permission: 
> actions=READ,WRITE,EXEC]  
>  ns1funct  ns1,ns1:tbl2,,: [Permission: 
> actions=READ,WRITE,EXEC,CREATE,ADMIN] 
>  hbase ns2,ns2:tbl1,,: [Permission: 
> actions=READ,WRITE,EXEC,CREATE,ADMIN] 
> 9 row(s) in 1.8090 seconds
> As you can see user test_user does not appear in the output, but on can see 
> the permission in the ACL table. 
> hbase(main):001:0>  scan 'hbase:acl'
> ROWCOLUMN+CELL
> 
>  @finance  column=l:sh82993, timestamp=105519510, 
> value=RWXCA 
>  @gcbcppdn column=l:hdfs, timestamp=1446141119602, 
> value=RWCXA
>  @hbasecolumn=l:hdfs, timestamp=1446141485136, 
> value=RWCAX
>  @ns1  column=l:@hbaseappltest_ns1admin, 
> timestamp=1447437007467, value=RWXCA 
>  @ns1  column=l:@hbaseappltest_ns1funct, 
> timestamp=1447427366835, value=RWX   
>  @ns2  column=l:@hbaseappltest_ns2admin, 
> timestamp=1446674470456, value=XCA   
>  @ns2  column=l:test_user, 
> timestamp=1447692840030, value=RWAC   
>  
>  @ns3  column=l:test_user, 
> timestamp=1447692860434, value=RWXAC  
>  
>  finance:emp   column=l:sh82993, timestamp=107723316, 
> value=RWXCA 
>  hbase:acl column=l:@hbaseglobaldba, 
> timestamp=1446590375370, value=XCA   
>  hbase:acl column=l:@hbaseglobaloper, 
> timestamp=1446590387965, value=XA   
>  hbase:acl column=l:hdfs, timestamp=1446141737213, 
> value=RWCAX
>  ns1:tbl1  column=l:sh82993, timestamp=1446674153058, 
> value=RWXCA 
>  ns1:tbl2  column=l:@hbaseappltest_ns1funct, 
> timestamp=1447183824580, value=RWX  

[jira] [Updated] (HBASE-14777) Replication fails with IndexOutOfBoundsException

[Ashu Pachauri (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashu Pachauri updated HBASE-14777:
--
Attachment: HBASE-14777-4.patch

V4:
Modified as per [~anoop.hbase]'s comment. Also incorporated suggested change 
from [~Bhupendra] and added a FailingDummyReplicator that throws an exception 
while shipping edits. 

> Replication fails with IndexOutOfBoundsException
> 
>
> Key: HBASE-14777
> URL: https://issues.apache.org/jira/browse/HBASE-14777
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Reporter: Bhupendra Kumar Jain
>Assignee: Ashu Pachauri
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14777-1.patch, HBASE-14777-2.patch, 
> HBASE-14777-3.patch, HBASE-14777-4.patch, HBASE-14777.patch
>
>
> Replication fails with IndexOutOfBoundsException 
> {code}
> regionserver.ReplicationSource$ReplicationSourceWorkerThread(939): 
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint
>  threw unknown exception:java.lang.IndexOutOfBoundsException: Index: 1, Size: 
> 1
>   at java.util.ArrayList.rangeCheck(Unknown Source)
>   at java.util.ArrayList.remove(Unknown Source)
>   at 
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicate(HBaseInterClusterReplicationEndpoint.java:222)
> {code}
> Its happening due to incorrect removal of entries from the replication 
> entries list. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14777) Replication fails with IndexOutOfBoundsException

[Ashu Pachauri (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashu Pachauri updated HBASE-14777:
--
Attachment: HBASE-14777-5.patch

V5: Using FailingDummyReplicator every alternate time slows the test down quite 
a lot; modifying it to fail only once.

> Replication fails with IndexOutOfBoundsException
> 
>
> Key: HBASE-14777
> URL: https://issues.apache.org/jira/browse/HBASE-14777
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Reporter: Bhupendra Kumar Jain
>Assignee: Ashu Pachauri
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14777-1.patch, HBASE-14777-2.patch, 
> HBASE-14777-3.patch, HBASE-14777-4.patch, HBASE-14777-5.patch, 
> HBASE-14777.patch
>
>
> Replication fails with IndexOutOfBoundsException 
> {code}
> regionserver.ReplicationSource$ReplicationSourceWorkerThread(939): 
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint
>  threw unknown exception:java.lang.IndexOutOfBoundsException: Index: 1, Size: 
> 1
>   at java.util.ArrayList.rangeCheck(Unknown Source)
>   at java.util.ArrayList.remove(Unknown Source)
>   at 
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicate(HBaseInterClusterReplicationEndpoint.java:222)
> {code}
> Its happening due to incorrect removal of entries from the replication 
> entries list. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14822) Renewing leases of scanners doesn't work

[Lars Hofhansl (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011955#comment-15011955
 ] 

Lars Hofhansl commented on HBASE-14822:
---

Thanks [~samarthjain]. Good point. Although I think we only have avoid the {{if 
(!moreResults || closeScanner)}} part.
This is a bit fragile now. I'm tempted to remove that feature for now until 
this is working and not fragile.

> Renewing leases of scanners doesn't work
> 
>
> Key: HBASE-14822
> URL: https://issues.apache.org/jira/browse/HBASE-14822
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.98.14
>Reporter: Samarth Jain
>Assignee: Lars Hofhansl
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17, 1.0.4
>
> Attachments: 14822-0.98-v2.txt, 14822-0.98-v3.txt, 14822-0.98-v4.txt, 
> 14822-0.98.txt
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14689) Addendum and unit test for HBASE-13471

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010432#comment-15010432
 ] 

Hudson commented on HBASE-14689:


FAILURE: Integrated in HBase-1.2-IT #289 (See 
[https://builds.apache.org/job/HBase-1.2-IT/289/])
Revert "HBASE-14689 Addendum and unit test for HBASE-13471" (enis: rev 
d6a2d7ab2d779847028c89a31bacd374610c5aa2)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java


> Addendum and unit test for HBASE-13471
> --
>
> Key: HBASE-14689
> URL: https://issues.apache.org/jira/browse/HBASE-14689
> Project: HBase
>  Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: hbase-14689_v1-branch-1.1.patch, 
> hbase-14689_v1-branch-1.1.patch, hbase-14689_v1.patch
>
>
> One of our customers ran into HBASE-13471, which resulted in all the handlers 
> getting blocked and various other issues. While backporting the issue, I 
> noticed that there is one more case where we might go into infinite loop. In 
> case a row lock cannot be acquired (due to a previous leak for example which 
> we have seen in Phoenix before) this will cause similar infinite loop. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13471) Fix a possible infinite loop in doMiniBatchMutation

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-13471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010433#comment-15010433
 ] 

Hudson commented on HBASE-13471:


FAILURE: Integrated in HBase-1.2-IT #289 (See 
[https://builds.apache.org/job/HBase-1.2-IT/289/])
Revert "HBASE-14689 Addendum and unit test for HBASE-13471" (enis: rev 
d6a2d7ab2d779847028c89a31bacd374610c5aa2)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java


> Fix a possible infinite loop in doMiniBatchMutation
> ---
>
> Key: HBASE-13471
> URL: https://issues.apache.org/jira/browse/HBASE-13471
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.0.0, 2.0.0, 1.1.0, 0.98.13
>Reporter: Elliott Clark
>Assignee: Rajesh Nishtala
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13471-v1.patch, HBASE-13471.patch
>
>
> {code}
> Thread 4139 
> (regionserver/hbase412.example.com/10.158.6.53:60020-splits-1429003183537):
>   State: WAITING
>   Blocked count: 131
>   Waited count: 228
>   Waiting on 
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@50714dc3
>   Stack:
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> 
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1371)
> org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1325)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:352)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:252)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:509)
> 
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:84)
> 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow

[Heng Chen (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010428#comment-15010428
 ] 

Heng Chen commented on HBASE-14703:
---

{quote}
Now, the timeout tracking is encapsulated and reused in the RetryingCaller and 
in the callable.
{quote}
I like the idea,  it is amazing!   And unify switching code in AP looks 
wonderful!
Base on that, i made some little changes.
 * Add some code in AP.run 
{code: title=AsyncProcess.java}
  res = caller.callWithoutRetries(callable, 
currentCallTotalTimeout);
  if (res == null|| res.getResults().size() == 0) {
if (currentCallable != null) {
  // As mutateRow, if statistic off, nothing will return. So 
let's stop.
  actionsInProgress.set(0); 
}
return;
  }
{code}

* Add some code in HTable.mutateRow
{code: title=HTable.java}
int remainingTime = tracker.getRemainingTime(callTimeout);
if (remainingTime == 0) {
  throw new DoNotRetryIOException("Timeout for mutate row");
}
{code}

 * remove some code unused in RpcRetryingCallerImpl
{code}
public T callWithoutRetries(RetryingCallable callable, int callTimeout)
  throws IOException, RuntimeException {
+tracker.start(); //Remove it.
{code}

 * fix some checkstyle errors and 
org.apache.hadoop.hbase.TestInterfaceAudienceAnnotations failed.


Let's say how QA talks.




> not collect stats when call HTable.mutateRow 
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
>  Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, 
> HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, HBASE-14703.patch, 
> HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, 
> HBASE-14703_v5.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update 
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}}  by 
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we 
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
>   @Override
>   public T callWithRetries(RetryingCallable callable, int callTimeout)
>   throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
>   }
>   @Override
>   public T callWithoutRetries(RetryingCallable callable, int callTimeout)
>   throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
>   }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do 
> update again. 
> {code}
> // update the stats about the region, if its a user table. We don't want to 
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
>   result = ResultStatsUtil.updateStats(result,
>   AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary,  remove it?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14777) Replication fails with IndexOutOfBoundsException

[Bhupendra Kumar Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010766#comment-15010766
 ] 

Bhupendra Kumar Jain commented on HBASE-14777:
--

Thanks Ashu and Appy
I was on festival vacation, So didn't check this further. 

V3 looks good. Can you also add the failure scenario in test 
case(DummyReplicator can throw exception). This can cover the scenarios when 
entry from entryLists should not be removed and retried. 

Initially I observed the index problem during my code review and during 
simulation of the same, I was printing the index returned from the future like 
below code
{code}
int i = (int)f.get();
entryLists.remove(i);
{code}
Since I was assigning as int , so thats the reason, I got 
IndexOutOfBoundsException. But as [~ashu210890] pointed out, before index 
problem, the other problem of removal exists. 

Thanks for looking into this

> Replication fails with IndexOutOfBoundsException
> 
>
> Key: HBASE-14777
> URL: https://issues.apache.org/jira/browse/HBASE-14777
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Reporter: Bhupendra Kumar Jain
>Assignee: Bhupendra Kumar Jain
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14777-1.patch, HBASE-14777-2.patch, 
> HBASE-14777-3.patch, HBASE-14777.patch
>
>
> Replication fails with IndexOutOfBoundsException 
> {code}
> regionserver.ReplicationSource$ReplicationSourceWorkerThread(939): 
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint
>  threw unknown exception:java.lang.IndexOutOfBoundsException: Index: 1, Size: 
> 1
>   at java.util.ArrayList.rangeCheck(Unknown Source)
>   at java.util.ArrayList.remove(Unknown Source)
>   at 
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicate(HBaseInterClusterReplicationEndpoint.java:222)
> {code}
> Its happening due to incorrect removal of entries from the replication 
> entries list. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11393) Replication TableCfs should be a PB object rather than a string

[Ashish Singhi (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-11393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010765#comment-15010765
 ] 

Ashish Singhi commented on HBASE-11393:
---

[~chenheng], can you upload the patch on review board ?

> Replication TableCfs should be a PB object rather than a string
> ---
>
> Key: HBASE-11393
> URL: https://issues.apache.org/jira/browse/HBASE-11393
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Enis Soztutar
> Fix For: 2.0.0
>
> Attachments: HBASE-11393.patch, HBASE-11393_v1.patch, 
> HBASE-11393_v2.patch, HBASE-11393_v3.patch, HBASE-11393_v4.patch, 
> HBASE-11393_v5.patch, HBASE-11393_v6.patch, HBASE-11393_v7.patch, 
> HBASE-11393_v8.patch, HBASE-11393_v9.patch
>
>
> We concatenate the list of tables and column families in format  
> "table1:cf1,cf2;table2:cfA,cfB" in zookeeper for table-cf to replication peer 
> mapping. 
> This results in ugly parsing code. We should do this a PB object. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14803) Add some debug logs to StoreFileScanner

[Anoop Sam John (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010823#comment-15010823
 ] 

Anoop Sam John commented on HBASE-14803:


Ah..  bad that we have to have these much logic for the log decision. Either 
handle the issue with the test in that test case (change the log level from 
Debug to Info for that test alone?- Some tests we handle this way I guess) or 
change to Trace level as suggested by Elliott.

> Add some debug logs to StoreFileScanner
> ---
>
> Key: HBASE-14803
> URL: https://issues.apache.org/jira/browse/HBASE-14803
> Project: HBase
>  Issue Type: Bug
>Reporter: Jean-Marc Spaggiari
>Assignee: Jean-Marc Spaggiari
>Priority: Minor
>  Labels: beginner
> Fix For: 1.2.0
>
> Attachments: HBASE-14803.v0-trunk.patch, HBASE-14803.v1-trunk.patch, 
> HBASE-14803.v2-trunk.patch, HBASE-14803.v3-trunk.patch, 
> HBASE-14803.v4-trunk.patch, HBASE-14803.v4-trunk.patch, 
> HBASE-14803.v5-trunk.patch
>
>
> To validate some behaviors I had to add some logs into StoreFileScanner.
> I think it can be interesting for other people looking for debuging. So 
> sharing the modifications here.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14828) Deprecate stateful REST scanners

[Jonathan Hsieh (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010853#comment-15010853
 ] 

Jonathan Hsieh commented on HBASE-14828:


-1.  Deprecate in 2.0 and keep in 1.x please.  Being more aggressive breaks
our compat/deprecation story, and unless stock 2.x is wire compat the
proxies are our shield.




-- 
// Jonathan Hsieh (shay)
// HBase Tech Lead, Software Engineer, Cloudera
// j...@cloudera.com // @jmhsieh


> Deprecate stateful REST scanners
> 
>
> Key: HBASE-14828
> URL: https://issues.apache.org/jira/browse/HBASE-14828
> Project: HBase
>  Issue Type: Task
>Reporter: Andrew Purtell
>
> Deprecate the REST gateway's stateful scanner API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14822) Renewing leases of scanners doesn't work

[Samarth Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Samarth Jain updated HBASE-14822:
-
Attachment: 14822-0.98-v3.txt

Looks like with with the changes made in v2 patch, scanners were getting 
removed because they were not returning any rows (which they are not supposed 
to). Attached patch fixes that.

> Renewing leases of scanners doesn't work
> 
>
> Key: HBASE-14822
> URL: https://issues.apache.org/jira/browse/HBASE-14822
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.98.14
>Reporter: Samarth Jain
>Assignee: Lars Hofhansl
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17, 1.0.4
>
> Attachments: 14822-0.98-v2.txt, 14822-0.98-v3.txt, 14822-0.98.txt
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14777) Replication fails with IndexOutOfBoundsException

[Bhupendra Kumar Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bhupendra Kumar Jain updated HBASE-14777:
-
Assignee: Ashu Pachauri  (was: Bhupendra Kumar Jain)

> Replication fails with IndexOutOfBoundsException
> 
>
> Key: HBASE-14777
> URL: https://issues.apache.org/jira/browse/HBASE-14777
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Reporter: Bhupendra Kumar Jain
>Assignee: Ashu Pachauri
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14777-1.patch, HBASE-14777-2.patch, 
> HBASE-14777-3.patch, HBASE-14777.patch
>
>
> Replication fails with IndexOutOfBoundsException 
> {code}
> regionserver.ReplicationSource$ReplicationSourceWorkerThread(939): 
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint
>  threw unknown exception:java.lang.IndexOutOfBoundsException: Index: 1, Size: 
> 1
>   at java.util.ArrayList.rangeCheck(Unknown Source)
>   at java.util.ArrayList.remove(Unknown Source)
>   at 
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicate(HBaseInterClusterReplicationEndpoint.java:222)
> {code}
> Its happening due to incorrect removal of entries from the replication 
> entries list. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14829) Add more checkstyles

[Jonathan Hsieh (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010851#comment-15010851
 ] 

Jonathan Hsieh commented on HBASE-14829:


Check style is run before and after the patch. the warning complains if
the delta goes the wrong way.




-- 
// Jonathan Hsieh (shay)
// HBase Tech Lead, Software Engineer, Cloudera
// j...@cloudera.com // @jmhsieh


> Add more checkstyles
> 
>
> Key: HBASE-14829
> URL: https://issues.apache.org/jira/browse/HBASE-14829
> Project: HBase
>  Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-14829-master.patch
>
>
> This jira will add following checkstyles:
> [ImportOrder|http://checkstyle.sourceforge.net/config_imports.html#ImportOrder]
>  : keep imports in sorted order
> [LeftCurly|http://checkstyle.sourceforge.net/config_blocks.html#LeftCurly] : 
> Placement of left curly brace. Does 'eol' sounds right setting?
> [NeedBraces|http://checkstyle.sourceforge.net/config_blocks.html#NeedBraces] 
> : braces around code blocks
> [JavadocTagContinuationIndentation|http://checkstyle.sourceforge.net/config_javadoc.html#JavadocTagContinuationIndentation]
>  : Avoid weird indentations in javadocs
> [NonEmptyAtclauseDescription|http://checkstyle.sourceforge.net/config_javadoc.html#NonEmptyAtclauseDescription]
>  : We have so many empty javadoc @ clauses. This'll take care of it.
>  
> [Indentation|http://checkstyle.sourceforge.net/config_misc.html#Indentation] 
> : Bad indentation hurts code readability. We have indentation guidelines, 
> should be fine enforcing them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14689) Addendum and unit test for HBASE-13471

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010772#comment-15010772
 ] 

Hudson commented on HBASE-14689:


SUCCESS: Integrated in HBase-1.3 #377 (See 
[https://builds.apache.org/job/HBase-1.3/377/])
Revert "HBASE-14689 Addendum and unit test for HBASE-13471" (enis: rev 
d4d3b1954c178d58fc9d58ade6df78ba941ca0a1)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java


> Addendum and unit test for HBASE-13471
> --
>
> Key: HBASE-14689
> URL: https://issues.apache.org/jira/browse/HBASE-14689
> Project: HBase
>  Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: hbase-14689_v1-branch-1.1.patch, 
> hbase-14689_v1-branch-1.1.patch, hbase-14689_v1.patch
>
>
> One of our customers ran into HBASE-13471, which resulted in all the handlers 
> getting blocked and various other issues. While backporting the issue, I 
> noticed that there is one more case where we might go into infinite loop. In 
> case a row lock cannot be acquired (due to a previous leak for example which 
> we have seen in Phoenix before) this will cause similar infinite loop. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13471) Fix a possible infinite loop in doMiniBatchMutation

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-13471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010773#comment-15010773
 ] 

Hudson commented on HBASE-13471:


SUCCESS: Integrated in HBase-1.3 #377 (See 
[https://builds.apache.org/job/HBase-1.3/377/])
Revert "HBASE-14689 Addendum and unit test for HBASE-13471" (enis: rev 
d4d3b1954c178d58fc9d58ade6df78ba941ca0a1)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java


> Fix a possible infinite loop in doMiniBatchMutation
> ---
>
> Key: HBASE-13471
> URL: https://issues.apache.org/jira/browse/HBASE-13471
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.0.0, 2.0.0, 1.1.0, 0.98.13
>Reporter: Elliott Clark
>Assignee: Rajesh Nishtala
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13471-v1.patch, HBASE-13471.patch
>
>
> {code}
> Thread 4139 
> (regionserver/hbase412.example.com/10.158.6.53:60020-splits-1429003183537):
>   State: WAITING
>   Blocked count: 131
>   Waited count: 228
>   Waiting on 
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@50714dc3
>   Stack:
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> 
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1371)
> org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1325)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:352)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:252)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:509)
> 
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:84)
> 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13471) Fix a possible infinite loop in doMiniBatchMutation

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-13471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010884#comment-15010884
 ] 

Hudson commented on HBASE-13471:


FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1134 (See 
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1134/])
Revert "HBASE-14689 Addendum and unit test for HBASE-13471" (enis: rev 
ea3c2e9064ca290d57c2bbbef6adf3cc89270138)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java


> Fix a possible infinite loop in doMiniBatchMutation
> ---
>
> Key: HBASE-13471
> URL: https://issues.apache.org/jira/browse/HBASE-13471
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.0.0, 2.0.0, 1.1.0, 0.98.13
>Reporter: Elliott Clark
>Assignee: Rajesh Nishtala
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13471-v1.patch, HBASE-13471.patch
>
>
> {code}
> Thread 4139 
> (regionserver/hbase412.example.com/10.158.6.53:60020-splits-1429003183537):
>   State: WAITING
>   Blocked count: 131
>   Waited count: 228
>   Waiting on 
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@50714dc3
>   Stack:
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> 
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1371)
> org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1325)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:352)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:252)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:509)
> 
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:84)
> 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010883#comment-15010883
 ] 

Hudson commented on HBASE-14584:


FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1134 (See 
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1134/])
HBASE-14584 TestNamespacesInstanceModel fails on jdk8 (apurtell: rev 
5227f80b01807bd83db264579f942833e26f81f6)
* hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/model/TestModelBase.java


> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
>  Issue Type: Test
>  Components: REST, test
>Reporter: Nick Dimiduk
>Assignee: Matt Warhaftig
> Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.17
>
> Attachments: 14584.00.patch, disable.txt, hbase-14584-v1.patch
>
>
> Noticed this in the [build 
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
>  of HBASE-12911. Seems this test has been failing for a long time, got all 
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I 
> guess tests usually fail in {{hbase-server}}, so we don't often get to 
> {{hbase-rest}} module.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14791) Batch Deletes in MapReduce jobs (0.98)

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010881#comment-15010881
 ] 

Hudson commented on HBASE-14791:


FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1134 (See 
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1134/])
HBASE-14791 Batch Deletes in MapReduce jobs (larsh: rev 
d25d74e121901d4d9705ae2c94256d947ad0a708)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestBufferedHTable.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/MultiTableOutputFormat.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/BufferedHTable.java


> Batch Deletes in MapReduce jobs (0.98)
> --
>
> Key: HBASE-14791
> URL: https://issues.apache.org/jira/browse/HBASE-14791
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.98.16
>Reporter: Lars Hofhansl
>Assignee: Alex Araujo
>  Labels: mapreduce
> Fix For: 0.98.17
>
> Attachments: HBASE-14791-0.98-v1.patch, HBASE-14791-0.98-v2.patch, 
> HBASE-14791-0.98.patch
>
>
> We found that some of our copy table job run for many hours, even when there 
> isn't that much data to copy.
> [~vik.karma] did his magic and found that the issue is with copying delete 
> markers (we use raw mode to also move deletes across).
> Looking at the code in 0.98 it's immediately obvious that deletes (unlike 
> puts) are not batched and hence sent to the other side one by one, causing a 
> network RTT for each delete marker.
> Looks like in trunk it's doing the right thing (using BufferedMutators for 
> all mutations in TableOutputFormat). So likely only a 0.98 (and 1.0, 1.1, 
> 1.2?) issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14689) Addendum and unit test for HBASE-13471

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010882#comment-15010882
 ] 

Hudson commented on HBASE-14689:


FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1134 (See 
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1134/])
Revert "HBASE-14689 Addendum and unit test for HBASE-13471" (enis: rev 
ea3c2e9064ca290d57c2bbbef6adf3cc89270138)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java


> Addendum and unit test for HBASE-13471
> --
>
> Key: HBASE-14689
> URL: https://issues.apache.org/jira/browse/HBASE-14689
> Project: HBase
>  Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: hbase-14689_v1-branch-1.1.patch, 
> hbase-14689_v1-branch-1.1.patch, hbase-14689_v1.patch
>
>
> One of our customers ran into HBASE-13471, which resulted in all the handlers 
> getting blocked and various other issues. While backporting the issue, I 
> noticed that there is one more case where we might go into infinite loop. In 
> case a row lock cannot be acquired (due to a previous leak for example which 
> we have seen in Phoenix before) this will cause similar infinite loop. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14824) HBaseAdmin.mergeRegions should use full region names instead of encoded region names

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010736#comment-15010736
 ] 

Hadoop QA commented on HBASE-14824:
---

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12772935/HBASE-14824-v2.patch
  against master branch at commit 5ebd7660a94bfb18e6e05b6e46195c76c099eda2.
  ATTACHMENT ID: 12772935

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 3 new 
or modified tests.

{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0 
2.7.1)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 protoc{color}.  The applied patch does not increase the 
total number of protoc compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any  new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16567//testReport/
Release Findbugs (version 2.0.3)warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16567//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16567//artifact/patchprocess/checkstyle-aggregate.html

  Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16567//console

This message is automatically generated.

> HBaseAdmin.mergeRegions should use full region names instead of encoded 
> region names
> 
>
> Key: HBASE-14824
> URL: https://issues.apache.org/jira/browse/HBASE-14824
> Project: HBase
>  Issue Type: Bug
>Reporter: Eungsop Yoo
>Priority: Minor
> Attachments: HBASE-14824-v1.patch, HBASE-14824-v2.patch, 
> HBASE-14824.patch
>
>
> HBaseAdmin.mergeRegions() calls HBaseAdmin.getRegion() internally. 
> HBaseAdmin.getRegion() requires the full region name. So 
> MetaTableAccessor.getRegion always returns null and this causes one more meta 
> table scan.
> {code}
>   Pair getRegion(final byte[] regionName) throws 
> IOException {
> if (regionName == null) {
>   throw new IllegalArgumentException("Pass a table name or region name");
> }
> Pair pair =
>   MetaTableAccessor.getRegion(connection, regionName);
> if (pair == null) {
> {code}
> I suppose to use full region names instead of encoded region names in 
> HBaseAdmin.mergeRegions().



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [jira] [Created] (HBASE-14828) Deprecate stateful REST scanners

[Jonathan Hsieh]

-1.  Deprecate in 2.0 and keep in 1.x please.  Being more aggressive breaks
our compat/deprecation story, and unless stock 2.x is wire compat the
proxies are our shield.

On Tuesday, November 17, 2015, Enis Soztutar (JIRA)  wrote:

>
> [
> https://issues.apache.org/jira/browse/HBASE-14828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010108#comment-15010108
> ]
>
> Enis Soztutar commented on HBASE-14828:
> ---
>
> bq. Deprecate in 1.x and remove in 2.x?
> +1. I think it is fine. 2.0 removes a lot of other stuff in the client
> APIs anyway.
>
> > Deprecate stateful REST scanners
> > 
> >
> > Key: HBASE-14828
> > URL: https://issues.apache.org/jira/browse/HBASE-14828
> > Project: HBase
> >  Issue Type: Task
> >Reporter: Andrew Purtell
> >
> > Deprecate the REST gateway's stateful scanner API.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>


-- 
// Jonathan Hsieh (shay)
// HBase Tech Lead, Software Engineer, Cloudera
// j...@cloudera.com // @jmhsieh

[jira] [Commented] (HBASE-14777) Replication fails with IndexOutOfBoundsException

[Anoop Sam John (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010814#comment-15010814
 ] 

Anoop Sam John commented on HBASE-14777:


nit:
bq.entryLists.remove((int)f.get());
f.get().intValue() can be a better choice?

Seeing the way we maintain these 2 lists of entryLists and futures, we maintain 
the same ordinal and not even we have to maintain the ordinal in the Runnable 
and return it may be.  The nth future item in futures List relates to nth item 
in entryList.  Ya for better understand ability of the code, let us not change 
that part.

+1 for the patch..  Great catch guys.

> Replication fails with IndexOutOfBoundsException
> 
>
> Key: HBASE-14777
> URL: https://issues.apache.org/jira/browse/HBASE-14777
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Reporter: Bhupendra Kumar Jain
>Assignee: Ashu Pachauri
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14777-1.patch, HBASE-14777-2.patch, 
> HBASE-14777-3.patch, HBASE-14777.patch
>
>
> Replication fails with IndexOutOfBoundsException 
> {code}
> regionserver.ReplicationSource$ReplicationSourceWorkerThread(939): 
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint
>  threw unknown exception:java.lang.IndexOutOfBoundsException: Index: 1, Size: 
> 1
>   at java.util.ArrayList.rangeCheck(Unknown Source)
>   at java.util.ArrayList.remove(Unknown Source)
>   at 
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicate(HBaseInterClusterReplicationEndpoint.java:222)
> {code}
> Its happening due to incorrect removal of entries from the replication 
> entries list. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010856#comment-15010856
 ] 

Hadoop QA commented on HBASE-14703:
---

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12772952/HBASE-14703_v5.1.patch
  against master branch at commit 5ebd7660a94bfb18e6e05b6e46195c76c099eda2.
  ATTACHMENT ID: 12772952

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 16 new 
or modified tests.

{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0 
2.7.1)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 protoc{color}.  The applied patch does not increase the 
total number of protoc compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any  new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   org.apache.hadoop.hbase.regionserver.TestHRegionOnCluster
  org.apache.hadoop.hbase.client.TestHCM
  org.apache.hadoop.hbase.quotas.TestQuotaThrottle

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16569//testReport/
Release Findbugs (version 2.0.3)warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16569//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16569//artifact/patchprocess/checkstyle-aggregate.html

  Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16569//console

This message is automatically generated.

> not collect stats when call HTable.mutateRow 
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
>  Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, 
> HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, HBASE-14703.patch, 
> HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, 
> HBASE-14703_v5.1.patch, HBASE-14703_v5.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update 
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}}  by 
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we 
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
>   @Override
>   public T callWithRetries(RetryingCallable callable, int callTimeout)
>   throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
>   }
>   @Override
>   public T callWithoutRetries(RetryingCallable callable, int callTimeout)
>   throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
>   }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do 
> update again. 
> {code}
> // update the stats about the region, if its a user table. We don't want to 
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
>   result = ResultStatsUtil.updateStats(result,
>   AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary,  remove it?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14791) Batch Deletes in MapReduce jobs (0.98)

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010921#comment-15010921
 ] 

Hudson commented on HBASE-14791:


FAILURE: Integrated in HBase-0.98-matrix #261 (See 
[https://builds.apache.org/job/HBase-0.98-matrix/261/])
HBASE-14791 Batch Deletes in MapReduce jobs (larsh: rev 
d25d74e121901d4d9705ae2c94256d947ad0a708)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/BufferedHTable.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/MultiTableOutputFormat.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestBufferedHTable.java


> Batch Deletes in MapReduce jobs (0.98)
> --
>
> Key: HBASE-14791
> URL: https://issues.apache.org/jira/browse/HBASE-14791
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.98.16
>Reporter: Lars Hofhansl
>Assignee: Alex Araujo
>  Labels: mapreduce
> Fix For: 0.98.17
>
> Attachments: HBASE-14791-0.98-v1.patch, HBASE-14791-0.98-v2.patch, 
> HBASE-14791-0.98.patch
>
>
> We found that some of our copy table job run for many hours, even when there 
> isn't that much data to copy.
> [~vik.karma] did his magic and found that the issue is with copying delete 
> markers (we use raw mode to also move deletes across).
> Looking at the code in 0.98 it's immediately obvious that deletes (unlike 
> puts) are not batched and hence sent to the other side one by one, causing a 
> network RTT for each delete marker.
> Looks like in trunk it's doing the right thing (using BufferedMutators for 
> all mutations in TableOutputFormat). So likely only a 0.98 (and 1.0, 1.1, 
> 1.2?) issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010923#comment-15010923
 ] 

Hudson commented on HBASE-14584:


FAILURE: Integrated in HBase-0.98-matrix #261 (See 
[https://builds.apache.org/job/HBase-0.98-matrix/261/])
HBASE-14584 TestNamespacesInstanceModel fails on jdk8 (apurtell: rev 
5227f80b01807bd83db264579f942833e26f81f6)
* hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/model/TestModelBase.java


> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
>  Issue Type: Test
>  Components: REST, test
>Reporter: Nick Dimiduk
>Assignee: Matt Warhaftig
> Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.17
>
> Attachments: 14584.00.patch, disable.txt, hbase-14584-v1.patch
>
>
> Noticed this in the [build 
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
>  of HBASE-12911. Seems this test has been failing for a long time, got all 
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I 
> guess tests usually fail in {{hbase-server}}, so we don't often get to 
> {{hbase-rest}} module.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13471) Fix a possible infinite loop in doMiniBatchMutation

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-13471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010924#comment-15010924
 ] 

Hudson commented on HBASE-13471:


FAILURE: Integrated in HBase-0.98-matrix #261 (See 
[https://builds.apache.org/job/HBase-0.98-matrix/261/])
Revert "HBASE-14689 Addendum and unit test for HBASE-13471" (enis: rev 
ea3c2e9064ca290d57c2bbbef6adf3cc89270138)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java


> Fix a possible infinite loop in doMiniBatchMutation
> ---
>
> Key: HBASE-13471
> URL: https://issues.apache.org/jira/browse/HBASE-13471
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.0.0, 2.0.0, 1.1.0, 0.98.13
>Reporter: Elliott Clark
>Assignee: Rajesh Nishtala
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13471-v1.patch, HBASE-13471.patch
>
>
> {code}
> Thread 4139 
> (regionserver/hbase412.example.com/10.158.6.53:60020-splits-1429003183537):
>   State: WAITING
>   Blocked count: 131
>   Waited count: 228
>   Waiting on 
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@50714dc3
>   Stack:
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> 
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1371)
> org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1325)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:352)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:252)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:509)
> 
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:84)
> 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14689) Addendum and unit test for HBASE-13471

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010922#comment-15010922
 ] 

Hudson commented on HBASE-14689:


FAILURE: Integrated in HBase-0.98-matrix #261 (See 
[https://builds.apache.org/job/HBase-0.98-matrix/261/])
Revert "HBASE-14689 Addendum and unit test for HBASE-13471" (enis: rev 
ea3c2e9064ca290d57c2bbbef6adf3cc89270138)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java


> Addendum and unit test for HBASE-13471
> --
>
> Key: HBASE-14689
> URL: https://issues.apache.org/jira/browse/HBASE-14689
> Project: HBase
>  Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: hbase-14689_v1-branch-1.1.patch, 
> hbase-14689_v1-branch-1.1.patch, hbase-14689_v1.patch
>
>
> One of our customers ran into HBASE-13471, which resulted in all the handlers 
> getting blocked and various other issues. While backporting the issue, I 
> noticed that there is one more case where we might go into infinite loop. In 
> case a row lock cannot be acquired (due to a previous leak for example which 
> we have seen in Phoenix before) this will cause similar infinite loop. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14803) Add some debug logs to StoreFileScanner

[Jean-Marc Spaggiari (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010830#comment-15010830
 ] 

Jean-Marc Spaggiari commented on HBASE-14803:
-

Hey, I'm planning to change that to trace and remove the logic. Goal is to have 
the ability to print some logs when required. Trace or debug doesn't make a big 
difference since it's the only log in this class. So it can still easily be 
activated... Will submit the update later today.

> Add some debug logs to StoreFileScanner
> ---
>
> Key: HBASE-14803
> URL: https://issues.apache.org/jira/browse/HBASE-14803
> Project: HBase
>  Issue Type: Bug
>Reporter: Jean-Marc Spaggiari
>Assignee: Jean-Marc Spaggiari
>Priority: Minor
>  Labels: beginner
> Fix For: 1.2.0
>
> Attachments: HBASE-14803.v0-trunk.patch, HBASE-14803.v1-trunk.patch, 
> HBASE-14803.v2-trunk.patch, HBASE-14803.v3-trunk.patch, 
> HBASE-14803.v4-trunk.patch, HBASE-14803.v4-trunk.patch, 
> HBASE-14803.v5-trunk.patch
>
>
> To validate some behaviors I had to add some logs into StoreFileScanner.
> I think it can be interesting for other people looking for debuging. So 
> sharing the modifications here.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [jira] [Created] (HBASE-14829) Add more checkstyles

[Jonathan Hsieh]

Check style is run before and after the patch. the warning complains if
the delta goes the wrong way.

On Tuesday, November 17, 2015, Appy (JIRA)  wrote:

>
> [
> https://issues.apache.org/jira/browse/HBASE-14829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010391#comment-15010391
> ]
>
> Appy commented on HBASE-14829:
> --
>
> No increase in checkstyle error! Something is wrong here. Will have a look
> tomorrow.
>
> > Add more checkstyles
> > 
> >
> > Key: HBASE-14829
> > URL: https://issues.apache.org/jira/browse/HBASE-14829
> > Project: HBase
> >  Issue Type: Improvement
> >Reporter: Appy
> >Assignee: Appy
> > Attachments: HBASE-14829-master.patch
> >
> >
> > This jira will add following checkstyles:
> > [ImportOrder|
> http://checkstyle.sourceforge.net/config_imports.html#ImportOrder] : keep
> imports in sorted order
> > [LeftCurly|
> http://checkstyle.sourceforge.net/config_blocks.html#LeftCurly] :
> Placement of left curly brace. Does 'eol' sounds right setting?
> > [NeedBraces|
> http://checkstyle.sourceforge.net/config_blocks.html#NeedBraces] : braces
> around code blocks
> > [JavadocTagContinuationIndentation|
> http://checkstyle.sourceforge.net/config_javadoc.html#JavadocTagContinuationIndentation]
> : Avoid weird indentations in javadocs
> > [NonEmptyAtclauseDescription|
> http://checkstyle.sourceforge.net/config_javadoc.html#NonEmptyAtclauseDescription]
> : We have so many empty javadoc @ clauses. This'll take care of it.
> >
> > [Indentation|
> http://checkstyle.sourceforge.net/config_misc.html#Indentation] : Bad
> indentation hurts code readability. We have indentation guidelines, should
> be fine enforcing them.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>


-- 
// Jonathan Hsieh (shay)
// HBase Tech Lead, Software Engineer, Cloudera
// j...@cloudera.com // @jmhsieh

[jira] [Commented] (HBASE-14777) Replication fails with IndexOutOfBoundsException

[Ashu Pachauri (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010453#comment-15010453
 ] 

Ashu Pachauri commented on HBASE-14777:
---

Actually, after re-reading the code, I am positive this is what is happening. 
We want to try to ship all edits in batches. The list of batches is entryLists. 
If a batch succeeds replicating, we immediately remove it from entryLists. 
After iterating through all futures and removing entries corresponding to 
successful futures, we take the "remaining" list and try again, till the 
entryLists becomes empty. 

> Replication fails with IndexOutOfBoundsException
> 
>
> Key: HBASE-14777
> URL: https://issues.apache.org/jira/browse/HBASE-14777
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Reporter: Bhupendra Kumar Jain
>Assignee: Bhupendra Kumar Jain
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14777-1.patch, HBASE-14777-2.patch, 
> HBASE-14777.patch
>
>
> Replication fails with IndexOutOfBoundsException 
> {code}
> regionserver.ReplicationSource$ReplicationSourceWorkerThread(939): 
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint
>  threw unknown exception:java.lang.IndexOutOfBoundsException: Index: 1, Size: 
> 1
>   at java.util.ArrayList.rangeCheck(Unknown Source)
>   at java.util.ArrayList.remove(Unknown Source)
>   at 
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicate(HBaseInterClusterReplicationEndpoint.java:222)
> {code}
> Its happening due to incorrect removal of entries from the replication 
> entries list. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14830) Fix broken links in 0.94 generated docs

[Appy (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010464#comment-15010464
 ] 

Appy commented on HBASE-14830:
--

Yeah, it's weird that it's parent and child are all public but 
BufferedDataBlockEncoder itself is private.
But excluding the whole path from javadoc does not seem to be the right fix. 
Either we should make BufferedDataBlockEncoder public or make the 4 classes 
referring it private. Since the 4 classes are being referred from within 
.../io/encoding only, and are implementations of what public DataBlockEncoding 
enum exposes, they don't really need to be public. So I think this would be the 
most appropriate thing to do.

> Fix broken links in 0.94 generated docs
> ---
>
> Key: HBASE-14830
> URL: https://issues.apache.org/jira/browse/HBASE-14830
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Affects Versions: 0.94.27
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Fix For: 0.94.28
>
> Attachments: HBASE-14830.patch
>
>
> {code}
> host: hbase.apache.org
> date: Mon, 16-Nov-2015 10:56:28 (local)
> Linklint version: 2.3.5_ingo_020
> #
> # ERROR   2 missing html files (cross referenced)
> #
> /0.94/devapidocs/org/apache/hadoop/hbase/avro/generated/HBase.html
> used in 1 file:
> /0.94/devapidocs/org/apache/hadoop/hbase/avro/package-summary.html
> /0.94/devapidocs/src-html/org/apache/hadoop/hbase/io/encoding/BufferedDataBlockEncoder.html
> used in 4 files:
> 
> /0.94/devapidocs/org/apache/hadoop/hbase/io/encoding/CopyKeyDataBlockEncoder.html
> 
> /0.94/devapidocs/org/apache/hadoop/hbase/io/encoding/DiffKeyDeltaEncoder.html
> 
> /0.94/devapidocs/org/apache/hadoop/hbase/io/encoding/FastDiffDeltaEncoder.html
> 
> /0.94/devapidocs/org/apache/hadoop/hbase/io/encoding/PrefixKeyDeltaEncoder.html
> #
> # ERROR   1 missing other file (cross referenced)
> #
> /0.94/devapidocs/org/apache/hadoop/hbase/HADOOP-1581
> used in 1 file:
> /0.94/devapidocs/org/apache/hadoop/hbase/HTableDescriptor.html
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14575) Reduce scope of compactions holding region lock

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010625#comment-15010625
 ] 

Hadoop QA commented on HBASE-14575:
---

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12772931/14575-v6.txt
  against master branch at commit 5ebd7660a94bfb18e6e05b6e46195c76c099eda2.
  ATTACHMENT ID: 12772931

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 3 new 
or modified tests.

{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0 
2.7.1)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 protoc{color}.  The applied patch does not increase the 
total number of protoc compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any  new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16566//testReport/
Release Findbugs (version 2.0.3)warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16566//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16566//artifact/patchprocess/checkstyle-aggregate.html

  Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16566//console

This message is automatically generated.

> Reduce scope of compactions holding region lock
> ---
>
> Key: HBASE-14575
> URL: https://issues.apache.org/jira/browse/HBASE-14575
> Project: HBase
>  Issue Type: Sub-task
>  Components: Compaction, regionserver
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: 14575-test-case.patch, 14575-v1.patch, 14575-v2.patch, 
> 14575-v3.patch, 14575-v4.patch, 14575-v5.patch, 14575-v6.txt, 14575-v6.txt, 
> 14575-v6.txt, 14575.v00.patch
>
>
> Per [~devaraj]'s idea on parent issue, let's see if we can reduce the scope 
> of critical section under which compactions hold the region read lock.
> Here is summary from parent issue:
> Another idea is we can reduce the scope of when the read lock is held during 
> compaction. In theory the compactor only needs a region read lock while 
> deciding what files to compact and at the time of committing the compaction. 
> We're protected from the case of region close events because compactions are 
> checking (every 10k bytes written) if the store has been closed in order to 
> abort in such a case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14823) HBase Ref Guide Refactoring

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010472#comment-15010472
 ] 

Hudson commented on HBASE-14823:


FAILURE: Integrated in HBase-Trunk_matrix #477 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/477/])
HBASE-14823 HBase Ref Guide Refactoring (mstanleyjones: rev 
623dc1303eee55610659f9e7e5a4a9149630adfe)
* src/main/asciidoc/_chapters/datamodel.adoc
* src/main/asciidoc/_chapters/appendix_acl_matrix.adoc
* src/main/asciidoc/_chapters/compression.adoc
* src/main/asciidoc/_chapters/security.adoc
* src/main/asciidoc/_chapters/hbase-default.adoc
* src/main/asciidoc/_chapters/appendix_hfile_format.adoc
* src/main/asciidoc/_chapters/hbase_history.adoc
* src/main/asciidoc/_chapters/other_info.adoc
* src/main/asciidoc/_chapters/getting_started.adoc
* src/main/asciidoc/_chapters/architecture.adoc
* src/main/asciidoc/_chapters/tracing.adoc
* src/main/asciidoc/_chapters/configuration.adoc
* src/main/asciidoc/_chapters/asf.adoc
* src/main/asciidoc/_chapters/zookeeper.adoc
* src/main/asciidoc/_chapters/performance.adoc
* src/main/asciidoc/_chapters/community.adoc
* src/main/asciidoc/_chapters/rpc.adoc
* src/main/asciidoc/_chapters/unit_testing.adoc
* src/main/asciidoc/_chapters/faq.adoc
* src/main/asciidoc/_chapters/schema_design.adoc
* src/main/asciidoc/_chapters/hbck_in_depth.adoc


> HBase Ref Guide Refactoring
> ---
>
> Key: HBASE-14823
> URL: https://issues.apache.org/jira/browse/HBASE-14823
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Affects Versions: 2.0.0
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Fix For: 2.0.0
>
> Attachments: HBASE-14823.patch
>
>
> Refactor some tables, links, and other things that don't look quite right in 
> the output.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14771) RpcServer#getRemoteAddress always returns null

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010471#comment-15010471
 ] 

Hudson commented on HBASE-14771:


FAILURE: Integrated in HBase-Trunk_matrix #477 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/477/])
HBASE-14771 RpcServer#getRemoteAddress always returns null (Abhishek (tedyu: 
rev 1b13bfcd43862451a807fa0f1943de781eef)
* hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/AbstractTestIPC.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java


> RpcServer#getRemoteAddress always returns null
> --
>
> Key: HBASE-14771
> URL: https://issues.apache.org/jira/browse/HBASE-14771
> Project: HBase
>  Issue Type: Bug
>  Components: IPC/RPC
>Affects Versions: 1.2.0
>Reporter: Abhishek Kumar
>Assignee: Abhishek Kumar
>Priority: Minor
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14771-V2.patch, HBASE-14771-V1.patch, 
> HBASE-14771-V2.patch, HBASE-14771.patch
>
>
> RpcServer.getRemoteAddress always returns null, because Call object is 
> getting initialized with null.This seems to be happening because of using 
> RpcServer.getRemoteIp() in  Call object constructor before RpcServer thread 
> local 'CurCall' being set in CallRunner.run method:
> {noformat}
> // --- RpcServer.java ---
> protected void processRequest(byte[] buf) throws IOException, 
> InterruptedException {
>  .
> // Call object getting initialized here with address 
> // obtained from RpcServer.getRemoteIp()
> Call call = new Call(id, this.service, md, header, param, cellScanner, this, 
> responder,
>   totalRequestSize, traceInfo, RpcServer.getRemoteIp());
>   scheduler.dispatch(new CallRunner(RpcServer.this, call));
>  }
> // getRemoteIp method gets address from threadlocal 'CurCall' which 
> // gets set in CallRunner.run and calling it before this as in above case, 
> will return null
> // --- CallRunner.java ---
> public void run() {
>   .   
>   Pair resultPair = null;
>   RpcServer.CurCall.set(call);
>   ..
> }
> // Using 'this.addr' in place of getRemoteIp method in RpcServer.java seems 
> to be fixing this issue
> Call call = new Call(id, this.service, md, header, param, cellScanner, this, 
> responder,
>   totalRequestSize, traceInfo, this.addr);
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14822) Renewing leases of scanners doesn't work

[Samarth Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010544#comment-15010544
 ] 

Samarth Jain commented on HBASE-14822:
--

Getting the same issue with the patch applied on latest of 0.98 branch. 
Relevant part of stacktrace:

Caused by: org.apache.hadoop.hbase.client.ScannerTimeoutException: 60073ms 
passed since the last invocation, timeout is currently set to 6
at 
org.apache.hadoop.hbase.client.ClientScanner.loadCache(ClientScanner.java:403)
at 
org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:338)
at 
org.apache.phoenix.iterate.ScanningResultIterator.next(ScanningResultIterator.java:55)
... 12 more
Caused by: org.apache.hadoop.hbase.UnknownScannerException: 
org.apache.hadoop.hbase.UnknownScannerException: Name: 37176, already closed?
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3222)
at 
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31068)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2150)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:104)
at 
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
at java.lang.Thread.run(Thread.java:745)

at sun.reflect.GeneratedConstructorAccessor59.newInstance(Unknown 
Source)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at 
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
at 
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
at 
org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:298)
at 
org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:214)
at 
org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:58)
at 
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:115)
at 
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:91)
at 
org.apache.hadoop.hbase.client.ClientScanner.loadCache(ClientScanner.java:385)
... 14 more
Caused by: 
org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.UnknownScannerException):
 org.apache.hadoop.hbase.UnknownScannerException: Name: 37176, already closed?
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3222)
at 
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31068)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2150)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:104)
at 
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
at java.lang.Thread.run(Thread.java:745)

at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1489)
at 
org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1691)
at 
org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1750)
at 
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:31514)
at 
org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:173)
... 18 more



> Renewing leases of scanners doesn't work
> 
>
> Key: HBASE-14822
> URL: https://issues.apache.org/jira/browse/HBASE-14822
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.98.14
>Reporter: Samarth Jain
>Assignee: Lars Hofhansl
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17, 1.0.4
>
> Attachments: 14822-0.98-v2.txt, 14822-0.98.txt
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14777) Replication fails with IndexOutOfBoundsException

[Ashu Pachauri (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010435#comment-15010435
 ] 

Ashu Pachauri commented on HBASE-14777:
---

[~appy] Thanks for the comment. But, now I am curious, because I am also not 
the one who wrote this piece of code originally :)  I had the impression that 
if f.get() failed, we would NOT want entryLists.remove to execute on that item, 
because we want to retry to ship the remaining entries in the list. Isn't that 
so?

> Replication fails with IndexOutOfBoundsException
> 
>
> Key: HBASE-14777
> URL: https://issues.apache.org/jira/browse/HBASE-14777
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Reporter: Bhupendra Kumar Jain
>Assignee: Bhupendra Kumar Jain
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14777-1.patch, HBASE-14777-2.patch, 
> HBASE-14777.patch
>
>
> Replication fails with IndexOutOfBoundsException 
> {code}
> regionserver.ReplicationSource$ReplicationSourceWorkerThread(939): 
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint
>  threw unknown exception:java.lang.IndexOutOfBoundsException: Index: 1, Size: 
> 1
>   at java.util.ArrayList.rangeCheck(Unknown Source)
>   at java.util.ArrayList.remove(Unknown Source)
>   at 
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicate(HBaseInterClusterReplicationEndpoint.java:222)
> {code}
> Its happening due to incorrect removal of entries from the replication 
> entries list. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13471) Fix a possible infinite loop in doMiniBatchMutation

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-13471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010444#comment-15010444
 ] 

Hudson commented on HBASE-13471:


FAILURE: Integrated in HBase-1.1-JDK8 #1685 (See 
[https://builds.apache.org/job/HBase-1.1-JDK8/1685/])
Revert "HBASE-14689 Addendum and unit test for HBASE-13471" (enis: rev 
214ea33d13a27de67b174e1fbeabb15e32b6943c)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java


> Fix a possible infinite loop in doMiniBatchMutation
> ---
>
> Key: HBASE-13471
> URL: https://issues.apache.org/jira/browse/HBASE-13471
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.0.0, 2.0.0, 1.1.0, 0.98.13
>Reporter: Elliott Clark
>Assignee: Rajesh Nishtala
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13471-v1.patch, HBASE-13471.patch
>
>
> {code}
> Thread 4139 
> (regionserver/hbase412.example.com/10.158.6.53:60020-splits-1429003183537):
>   State: WAITING
>   Blocked count: 131
>   Waited count: 228
>   Waiting on 
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@50714dc3
>   Stack:
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> 
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1371)
> org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1325)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:352)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:252)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:509)
> 
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:84)
> 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14689) Addendum and unit test for HBASE-13471

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010443#comment-15010443
 ] 

Hudson commented on HBASE-14689:


FAILURE: Integrated in HBase-1.1-JDK8 #1685 (See 
[https://builds.apache.org/job/HBase-1.1-JDK8/1685/])
Revert "HBASE-14689 Addendum and unit test for HBASE-13471" (enis: rev 
214ea33d13a27de67b174e1fbeabb15e32b6943c)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java


> Addendum and unit test for HBASE-13471
> --
>
> Key: HBASE-14689
> URL: https://issues.apache.org/jira/browse/HBASE-14689
> Project: HBase
>  Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: hbase-14689_v1-branch-1.1.patch, 
> hbase-14689_v1-branch-1.1.patch, hbase-14689_v1.patch
>
>
> One of our customers ran into HBASE-13471, which resulted in all the handlers 
> getting blocked and various other issues. While backporting the issue, I 
> noticed that there is one more case where we might go into infinite loop. In 
> case a row lock cannot be acquired (due to a previous leak for example which 
> we have seen in Phoenix before) this will cause similar infinite loop. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14825) HBase Ref Guide corrections of typos/misspellings

[Daniel Vimont (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010467#comment-15010467
 ] 

Daniel Vimont commented on HBASE-14825:
---

Well, I'm getting close to shutting down operations for the day, so tomorrow I 
will:
(1) See about reconciling the changes that I've made with the changes pushed 
today into the master branch from HBASE-14823 -- I think some of the same files 
were modified in dealing with both issues (such a reconciliation process is 
something I've not done before -- should be a worthwhile learning experience).
(2) Put it all together in a new patch submission (my first to the project).

> HBase Ref Guide corrections of typos/misspellings
> -
>
> Key: HBASE-14825
> URL: https://issues.apache.org/jira/browse/HBASE-14825
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Affects Versions: 2.0.0
>Reporter: Daniel Vimont
>Assignee: Daniel Vimont
>Priority: Minor
> Fix For: 2.0.0
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> Found the following list of typos/misspellings on the book.html page, and 
> thought I would make corrections to the appropriate src/main/asciidoc files 
> in which they are located. (This is just a good opportunity for me to become 
> familiar with submission of fixes/patches as a prelude to beginning to make 
> some coding contributions. This is also my first submission to the JIRA 
> system, so corrections to content/conventions are welcome!)
> [Note: I see that [~misty]  may be in the midst of a reformatting task -- 
> HBASE-14823 --  that might involve these same asciidoc files. Please advise 
> if I should wait on this task to avoid a possibly cumbersome Git 
> reconciliation mess. (?)]
> Here is the list of typos/misspellings. The format of each item is (a) the 
> problem is presented in brackets on the first line, and (b) the phrase (as it 
> currently appears in the text) is on the second line.
> ===
> ["you" should be "your", and "Kimballs'" should be "Kimball's" (move the 
> apostrophe) in the following:]
> A useful read setting config on you hadoop cluster is Aaron Kimballs' 
> Configuration Parameters: What can you just ignore?
> [Period needed after "a"]
> a.k.a pseudo-distributed
> ["empty" is misspelled]
> The default value in this configuration has been intentionally left emtpy in 
> order to honor the old hbase.regionserver.global.memstore.upperLimit property 
> if present.
> [All occurrences of "a HBase" should be changed to "an HBase" -- 15 
> occurrences found]
> ["file path are" should be "file paths are"]
> By default, all of HBase's ZooKeeper file path are configured with a relative 
> path, so they will all go under this directory unless changed.
> ["times" -- plural required]
> How many time to retry attempting to write a version file before just 
> aborting. 
> ["separated" is misspelled]
> Each attempt is seperated by the hbase.server.thread.wakefrequency 
> milliseconds.
> [space needed after quotation mark (include"limit)]
> Because this limit represents the "automatic include"limit...
> [space needed ("ashbase:metadata" should be "as hbase:metadata")]
> This helps to keep compaction of lean tables (such ashbase:meta) fast.
> [Acronym "ide" should be capitalized for clarity: IDE]
> Setting this to true can be useful in contexts other than the other side of a 
> maven generation; i.e. running in an ide. 
> [RuntimeException missing an "e"]
> You'll want to set this boolean to true to avoid seeing the RuntimException 
> complaint:
> [Space missing after "secure"]
> FS Permissions for the root directory in a secure(kerberos) setup.
> ["mutations" misspelled]
> ...will be created which will tail the logs and replicate the mutatations to 
> region replicas for tables that have region replication > 1.
> ["it such that" should be "is such that"]
> If your working set it such that block cache does you no good...
> ["an" should be "and"]
> See the Deveraj Das an Nicolas Liochon blog post...
> [Tag "" should be ""]
> hbase.coprocessor.master.classes
> [Misspelling of "implementations"]
> Those consumers are coprocessors, phoenix, replication endpoint 
> implemnetations or similar.
> [Misspelling of "cluster"]
> On upgrade, before running a rolling restart over the cluser...
> ["effect" should be "affect"]
> If NOT using BucketCache, this change does not effect you.
> [Need space after "throw"]
> This will throw`java.lang.NoSuchMethodError...
> ["habasee" should be "hbase"]
> You can pass commands to the HBase Shell in non-interactive mode (see 
> hbasee.shell.noninteractive)...
> ["ie" should be "i.e."]
> Restrict the amount of resources (ie regions, tables) a namespace can consume.
> ["an" should be "and"]
> ...but can be conjured on the fly while 

[jira] [Updated] (HBASE-14777) Replication fails with IndexOutOfBoundsException

[Ashu Pachauri (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashu Pachauri updated HBASE-14777:
--
Attachment: HBASE-14777-3.patch

V3: Added comments config changes to the Test.

> Replication fails with IndexOutOfBoundsException
> 
>
> Key: HBASE-14777
> URL: https://issues.apache.org/jira/browse/HBASE-14777
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Reporter: Bhupendra Kumar Jain
>Assignee: Bhupendra Kumar Jain
>Priority: Critical
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14777-1.patch, HBASE-14777-2.patch, 
> HBASE-14777-3.patch, HBASE-14777.patch
>
>
> Replication fails with IndexOutOfBoundsException 
> {code}
> regionserver.ReplicationSource$ReplicationSourceWorkerThread(939): 
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint
>  threw unknown exception:java.lang.IndexOutOfBoundsException: Index: 1, Size: 
> 1
>   at java.util.ArrayList.rangeCheck(Unknown Source)
>   at java.util.ArrayList.remove(Unknown Source)
>   at 
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.replicate(HBaseInterClusterReplicationEndpoint.java:222)
> {code}
> Its happening due to incorrect removal of entries from the replication 
> entries list. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14689) Addendum and unit test for HBASE-13471

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010473#comment-15010473
 ] 

Hudson commented on HBASE-14689:


FAILURE: Integrated in HBase-Trunk_matrix #477 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/477/])
Revert "HBASE-14689 Addendum and unit test for HBASE-13471" (enis: rev 
5ebd7660a94bfb18e6e05b6e46195c76c099eda2)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java


> Addendum and unit test for HBASE-13471
> --
>
> Key: HBASE-14689
> URL: https://issues.apache.org/jira/browse/HBASE-14689
> Project: HBase
>  Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: hbase-14689_v1-branch-1.1.patch, 
> hbase-14689_v1-branch-1.1.patch, hbase-14689_v1.patch
>
>
> One of our customers ran into HBASE-13471, which resulted in all the handlers 
> getting blocked and various other issues. While backporting the issue, I 
> noticed that there is one more case where we might go into infinite loop. In 
> case a row lock cannot be acquired (due to a previous leak for example which 
> we have seen in Phoenix before) this will cause similar infinite loop. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13471) Fix a possible infinite loop in doMiniBatchMutation

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-13471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010474#comment-15010474
 ] 

Hudson commented on HBASE-13471:


FAILURE: Integrated in HBase-Trunk_matrix #477 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/477/])
Revert "HBASE-14689 Addendum and unit test for HBASE-13471" (enis: rev 
5ebd7660a94bfb18e6e05b6e46195c76c099eda2)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java


> Fix a possible infinite loop in doMiniBatchMutation
> ---
>
> Key: HBASE-13471
> URL: https://issues.apache.org/jira/browse/HBASE-13471
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.0.0, 2.0.0, 1.1.0, 0.98.13
>Reporter: Elliott Clark
>Assignee: Rajesh Nishtala
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13471-v1.patch, HBASE-13471.patch
>
>
> {code}
> Thread 4139 
> (regionserver/hbase412.example.com/10.158.6.53:60020-splits-1429003183537):
>   State: WAITING
>   Blocked count: 131
>   Waited count: 228
>   Waiting on 
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@50714dc3
>   Stack:
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> 
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1371)
> org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1325)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:352)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:252)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:509)
> 
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:84)
> 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010546#comment-15010546
 ] 

Hadoop QA commented on HBASE-14703:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12772936/HBASE-14703_v5.patch
  against master branch at commit 5ebd7660a94bfb18e6e05b6e46195c76c099eda2.
  ATTACHMENT ID: 12772936

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 16 new 
or modified tests.

{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0 
2.7.1)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 protoc{color}.  The applied patch does not increase the 
total number of protoc compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:red}-1 checkstyle{color}.  The applied patch generated 
1728 checkstyle errors (more than the master's current 1727 errors).

{color:green}+1 findbugs{color}.  The patch does not introduce any  new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   org.apache.hadoop.hbase.client.TestHBaseAdminNoCluster

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16568//testReport/
Release Findbugs (version 2.0.3)warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16568//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16568//artifact/patchprocess/checkstyle-aggregate.html

Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16568//console

This message is automatically generated.

> not collect stats when call HTable.mutateRow 
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
>  Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, 
> HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, HBASE-14703.patch, 
> HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, 
> HBASE-14703_v5.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update 
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}}  by 
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we 
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
>   @Override
>   public T callWithRetries(RetryingCallable callable, int callTimeout)
>   throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
>   }
>   @Override
>   public T callWithoutRetries(RetryingCallable callable, int callTimeout)
>   throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
>   }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do 
> update again. 
> {code}
> // update the stats about the region, if its a user table. We don't want to 
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
>   result = ResultStatsUtil.updateStats(result,
>   AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary,  remove it?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14822) Renewing leases of scanners doesn't work

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010633#comment-15010633
 ] 

Hadoop QA commented on HBASE-14822:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12772929/14822-0.98-v2.txt
  against 0.98 branch at commit 5ebd7660a94bfb18e6e05b6e46195c76c099eda2.
  ATTACHMENT ID: 12772929

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 5 new 
or modified tests.

{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0 
2.7.1)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 protoc{color}.  The applied patch does not increase the 
total number of protoc compiler warnings.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 
28 warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any  new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   
org.apache.hadoop.hbase.regionserver.TestRegionServerMetrics

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16565//testReport/
Release Findbugs (version 2.0.3)warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16565//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16565//artifact/patchprocess/checkstyle-aggregate.html

  Javadoc warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16565//artifact/patchprocess/patchJavadocWarnings.txt
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16565//console

This message is automatically generated.

> Renewing leases of scanners doesn't work
> 
>
> Key: HBASE-14822
> URL: https://issues.apache.org/jira/browse/HBASE-14822
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.98.14
>Reporter: Samarth Jain
>Assignee: Lars Hofhansl
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17, 1.0.4
>
> Attachments: 14822-0.98-v2.txt, 14822-0.98.txt
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14825) HBase Ref Guide corrections of typos/misspellings

[Daniel Vimont (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010449#comment-15010449
 ] 

Daniel Vimont commented on HBASE-14825:
---

Okay, a couple more that I found while fixing the others:

http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/util/Writables.html#getHRegionInfo%28byte%29[Writables]
 <<-- This was another square-brackets problem.

hhttp://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Table.html#getRegionLocation(byte)[Table.getRegionLocation]
 <<-- removed the extra "h" from the beginning

=
And here is one that I could NOT resolve: it appears to be a valid URL that is 
being pulled from {{hbase-default.xml}}, and asciidoc appears to simply be 
dropping the final close-parenthesis from the link:

http://docs.oracle.com/javase/1.5.0/docs/api/java/net/Socket.html#getTcpNoDelay()

(Curious: viewing it here in "preview" mode, I see that JIRA is also dropping 
the final ")" from the link!)

And, FINALLY (I hope!!), it appears that backticks around a URL do NOT suffice 
to turn it into a literal (despite the asciidocs documentation apparently 
indicating that is SHOULD).  So I had to employ the "pass[]" option to get the 
following to appear as literals on {{book.html}}:

http://REGIONSERVER_HOSTNAME:60030
http://REGIONSERVER_HOSTNAME:60030/jmx?description=true
http://localhost:60010
http://REGIONSERVER_HOSTNAME:60010/jmx?description=true
http://people.apache.org/~YOU
http://master.example.org:16010
http://example.com:8000

> HBase Ref Guide corrections of typos/misspellings
> -
>
> Key: HBASE-14825
> URL: https://issues.apache.org/jira/browse/HBASE-14825
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Affects Versions: 2.0.0
>Reporter: Daniel Vimont
>Assignee: Daniel Vimont
>Priority: Minor
> Fix For: 2.0.0
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> Found the following list of typos/misspellings on the book.html page, and 
> thought I would make corrections to the appropriate src/main/asciidoc files 
> in which they are located. (This is just a good opportunity for me to become 
> familiar with submission of fixes/patches as a prelude to beginning to make 
> some coding contributions. This is also my first submission to the JIRA 
> system, so corrections to content/conventions are welcome!)
> [Note: I see that [~misty]  may be in the midst of a reformatting task -- 
> HBASE-14823 --  that might involve these same asciidoc files. Please advise 
> if I should wait on this task to avoid a possibly cumbersome Git 
> reconciliation mess. (?)]
> Here is the list of typos/misspellings. The format of each item is (a) the 
> problem is presented in brackets on the first line, and (b) the phrase (as it 
> currently appears in the text) is on the second line.
> ===
> ["you" should be "your", and "Kimballs'" should be "Kimball's" (move the 
> apostrophe) in the following:]
> A useful read setting config on you hadoop cluster is Aaron Kimballs' 
> Configuration Parameters: What can you just ignore?
> [Period needed after "a"]
> a.k.a pseudo-distributed
> ["empty" is misspelled]
> The default value in this configuration has been intentionally left emtpy in 
> order to honor the old hbase.regionserver.global.memstore.upperLimit property 
> if present.
> [All occurrences of "a HBase" should be changed to "an HBase" -- 15 
> occurrences found]
> ["file path are" should be "file paths are"]
> By default, all of HBase's ZooKeeper file path are configured with a relative 
> path, so they will all go under this directory unless changed.
> ["times" -- plural required]
> How many time to retry attempting to write a version file before just 
> aborting. 
> ["separated" is misspelled]
> Each attempt is seperated by the hbase.server.thread.wakefrequency 
> milliseconds.
> [space needed after quotation mark (include"limit)]
> Because this limit represents the "automatic include"limit...
> [space needed ("ashbase:metadata" should be "as hbase:metadata")]
> This helps to keep compaction of lean tables (such ashbase:meta) fast.
> [Acronym "ide" should be capitalized for clarity: IDE]
> Setting this to true can be useful in contexts other than the other side of a 
> maven generation; i.e. running in an ide. 
> [RuntimeException missing an "e"]
> You'll want to set this boolean to true to avoid seeing the RuntimException 
> complaint:
> [Space missing after "secure"]
> FS Permissions for the root directory in a secure(kerberos) setup.
> ["mutations" misspelled]
> ...will be created which will tail the logs and replicate the mutatations to 
> region replicas for tables that have region replication > 1.
> ["it such that" should be "is such that"]
> If your working set it such 

[jira] [Commented] (HBASE-13471) Fix a possible infinite loop in doMiniBatchMutation

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-13471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010452#comment-15010452
 ] 

Hudson commented on HBASE-13471:


FAILURE: Integrated in HBase-1.1-JDK7 #1598 (See 
[https://builds.apache.org/job/HBase-1.1-JDK7/1598/])
Revert "HBASE-14689 Addendum and unit test for HBASE-13471" (enis: rev 
214ea33d13a27de67b174e1fbeabb15e32b6943c)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java


> Fix a possible infinite loop in doMiniBatchMutation
> ---
>
> Key: HBASE-13471
> URL: https://issues.apache.org/jira/browse/HBASE-13471
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.0.0, 2.0.0, 1.1.0, 0.98.13
>Reporter: Elliott Clark
>Assignee: Rajesh Nishtala
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13471-v1.patch, HBASE-13471.patch
>
>
> {code}
> Thread 4139 
> (regionserver/hbase412.example.com/10.158.6.53:60020-splits-1429003183537):
>   State: WAITING
>   Blocked count: 131
>   Waited count: 228
>   Waiting on 
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@50714dc3
>   Stack:
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> 
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1371)
> org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1325)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:352)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:252)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:509)
> 
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:84)
> 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14689) Addendum and unit test for HBASE-13471

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010451#comment-15010451
 ] 

Hudson commented on HBASE-14689:


FAILURE: Integrated in HBase-1.1-JDK7 #1598 (See 
[https://builds.apache.org/job/HBase-1.1-JDK7/1598/])
Revert "HBASE-14689 Addendum and unit test for HBASE-13471" (enis: rev 
214ea33d13a27de67b174e1fbeabb15e32b6943c)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java


> Addendum and unit test for HBASE-13471
> --
>
> Key: HBASE-14689
> URL: https://issues.apache.org/jira/browse/HBASE-14689
> Project: HBase
>  Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: hbase-14689_v1-branch-1.1.patch, 
> hbase-14689_v1-branch-1.1.patch, hbase-14689_v1.patch
>
>
> One of our customers ran into HBASE-13471, which resulted in all the handlers 
> getting blocked and various other issues. While backporting the issue, I 
> noticed that there is one more case where we might go into infinite loop. In 
> case a row lock cannot be acquired (due to a previous leak for example which 
> we have seen in Phoenix before) this will cause similar infinite loop. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13471) Fix a possible infinite loop in doMiniBatchMutation

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-13471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010482#comment-15010482
 ] 

Hudson commented on HBASE-13471:


SUCCESS: Integrated in HBase-1.2 #381 (See 
[https://builds.apache.org/job/HBase-1.2/381/])
Revert "HBASE-14689 Addendum and unit test for HBASE-13471" (enis: rev 
d6a2d7ab2d779847028c89a31bacd374610c5aa2)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java


> Fix a possible infinite loop in doMiniBatchMutation
> ---
>
> Key: HBASE-13471
> URL: https://issues.apache.org/jira/browse/HBASE-13471
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.0.0, 2.0.0, 1.1.0, 0.98.13
>Reporter: Elliott Clark
>Assignee: Rajesh Nishtala
> Fix For: 2.0.0, 1.1.0, 0.98.13, 1.0.2
>
> Attachments: HBASE-13471-v1.patch, HBASE-13471.patch
>
>
> {code}
> Thread 4139 
> (regionserver/hbase412.example.com/10.158.6.53:60020-splits-1429003183537):
>   State: WAITING
>   Blocked count: 131
>   Waited count: 228
>   Waiting on 
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@50714dc3
>   Stack:
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> 
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1371)
> org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1325)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:352)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:252)
> 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:509)
> 
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:84)
> 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14689) Addendum and unit test for HBASE-13471

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010481#comment-15010481
 ] 

Hudson commented on HBASE-14689:


SUCCESS: Integrated in HBase-1.2 #381 (See 
[https://builds.apache.org/job/HBase-1.2/381/])
Revert "HBASE-14689 Addendum and unit test for HBASE-13471" (enis: rev 
d6a2d7ab2d779847028c89a31bacd374610c5aa2)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java


> Addendum and unit test for HBASE-13471
> --
>
> Key: HBASE-14689
> URL: https://issues.apache.org/jira/browse/HBASE-14689
> Project: HBase
>  Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: hbase-14689_v1-branch-1.1.patch, 
> hbase-14689_v1-branch-1.1.patch, hbase-14689_v1.patch
>
>
> One of our customers ran into HBASE-13471, which resulted in all the handlers 
> getting blocked and various other issues. While backporting the issue, I 
> noticed that there is one more case where we might go into infinite loop. In 
> case a row lock cannot be acquired (due to a previous leak for example which 
> we have seen in Phoenix before) this will cause similar infinite loop. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14703) not collect stats when call HTable.mutateRow

[Heng Chen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Heng Chen updated HBASE-14703:
--
Attachment: HBASE-14703_v5.1.patch

Fix bug about failed test case.
It is here
{code: title=RpcRetryingCallerImpl.java}
private long singleCallDuration(final long expectedSleep) {
  -return (EnvironmentEdgeManager.currentTime() - this.globalStartTime) + 
expectedSleep;
  +return (EnvironmentEdgeManager.currentTime() - tracker.getStartTime()) + 
expectedSleep;
}
{code}




> not collect stats when call HTable.mutateRow 
> -
>
> Key: HBASE-14703
> URL: https://issues.apache.org/jira/browse/HBASE-14703
> Project: HBase
>  Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, 
> HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, HBASE-14703.patch, 
> HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, 
> HBASE-14703_v5.1.patch, HBASE-14703_v5.patch
>
>
> In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update 
> serverStatistics twice.
> The first one is that we wrapper {{RetryingCallable}}  by 
> {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we 
> call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
> {code}
>   @Override
>   public T callWithRetries(RetryingCallable callable, int callTimeout)
>   throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
>   }
>   @Override
>   public T callWithoutRetries(RetryingCallable callable, int callTimeout)
>   throws IOException, RuntimeException {
> T result = delegate.callWithRetries(callable, callTimeout);
> return updateStatsAndUnwrap(result, callable);
>   }
> {code}
> The secondary one is after we get response, in {{receiveMultiAction}}, we do 
> update again. 
> {code}
> // update the stats about the region, if its a user table. We don't want to 
> slow down
> // updates to meta tables, especially from internal updates (master, etc).
> if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
>   result = ResultStatsUtil.updateStats(result,
>   AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
> }
> {code}
> It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary,  remove it?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14822) Renewing leases of scanners doesn't work

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010997#comment-15010997
 ] 

Hadoop QA commented on HBASE-14822:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12772969/14822-0.98-v3.txt
  against 0.98 branch at commit 5ebd7660a94bfb18e6e05b6e46195c76c099eda2.
  ATTACHMENT ID: 12772969

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 5 new 
or modified tests.

{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0 
2.7.1)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 protoc{color}.  The applied patch does not increase the 
total number of protoc compiler warnings.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 
28 warning messages.

{color:red}-1 checkstyle{color}.  The applied patch generated 
3899 checkstyle errors (more than the master's current 3898 errors).

{color:green}+1 findbugs{color}.  The patch does not introduce any  new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+// Stop collecting results if maxScannerResultSize is 
set and we have exceeded it
+
TEST_UTIL.getConfiguration().setInt(HConstants.HBASE_CLIENT_SCANNER_TIMEOUT_PERIOD,
 leaseTimeout);

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   org.apache.hadoop.hbase.mapreduce.TestImportExport

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16570//testReport/
Release Findbugs (version 2.0.3)warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16570//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16570//artifact/patchprocess/checkstyle-aggregate.html

Javadoc warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16570//artifact/patchprocess/patchJavadocWarnings.txt
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/16570//console

This message is automatically generated.

> Renewing leases of scanners doesn't work
> 
>
> Key: HBASE-14822
> URL: https://issues.apache.org/jira/browse/HBASE-14822
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.98.14
>Reporter: Samarth Jain
>Assignee: Lars Hofhansl
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17, 1.0.4
>
> Attachments: 14822-0.98-v2.txt, 14822-0.98-v3.txt, 14822-0.98.txt
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable

[Pankaj Kumar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011125#comment-15011125
 ] 

Pankaj Kumar commented on HBASE-14498:
--

Yeah [~busbey], this problem exist in earlier versions and 1.1+ also. 
Severity doesn't matter, we should fix this. Feel free to modify this  :) 

> Master stuck in infinite loop when all Zookeeper servers are unreachable
> 
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
>  Issue Type: Bug
>  Components: master
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Blocker
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4
>
> Attachments: HBASE-14498-V2.patch, HBASE-14498-V3.patch, 
> HBASE-14498-V4.patch, HBASE-14498.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to 
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO 
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)] 
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in 
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and 
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO 
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)] 
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)] 
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN 
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)] 
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO 
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)] 
> zookeeper.ClientCnxn: Opening socket connection to server 
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown 
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)] 
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)] 
> zookeeper.ClientCnxn: Opening socket connection to server 
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown 
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)] 
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in 
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and 
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000] 
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, 
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181, 
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException: 
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)] 
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)] 
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)] 
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181. 
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt 
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an 
> > active master. 
> > HM2 is keep on waiting for region server to report him as part of active 
> > master intialization.
> {noformat} 
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting 
> for region servers count to settle; currently checked in 0, slept for 0 ms, 
> expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval 
> of 1500 ms. | 
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> ---
> ---
> 2015-09-26 15:32:50,841 | INFO | HM2-Host:21300.activeMasterManager | Waiting 
> for region servers count to settle; currently checked in 0, slept for 483913 
> ms, expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, 
> interval of 1500 ms. | 
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> {noformat}
> > At other end, region servers are reporting to HM1 on 3 sec interval. Here 
> > region server retrieve master location from zookeeper only when they 
> > couldn't connect to Master (ServiceException).
> Region Server will not report HM2 as per current design until unless HM1 
> abort,so HM2 will 

[jira] [Commented] (HBASE-14791) Batch Deletes in MapReduce jobs (0.98)

[Alex Araujo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011123#comment-15011123
 ] 

Alex Araujo commented on HBASE-14791:
-

Those build failures look unrelated. HBase-0.98-on-Hadoop-1.1 has been failing 
with a compilation error since 
#[1132|https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1132/]:

{noformat}
[ERROR] 
/home/jenkins/jenkins-slave/workspace/HBase-0.98-on-Hadoop-1.1/hbase-common/src/main/java/org/apache/hadoop/hbase/security/UserProvider.java:[70,49]
 error: cannot find symbol
{noformat}

HBase-0.98-matrix appears to have unrelated test failures. Most of the failures 
have been around since 
#[258|https://builds.apache.org/job/HBase-0.98-matrix/258/]. These are new, but 
unrelated:

*jdk=latest1.7,label=Hadoop*

java.lang.ExceptionInInitializerError: null
at 
org.apache.hadoop.hbase.HBaseConfiguration.checkDefaultsVersion(HBaseConfiguration.java:73)

hbase-default.xml file seems to be for and old version of HBase (0.98.16), this 
version is 0.98.17-SNAPSHOT

java.lang.ExceptionInInitializerError: null
at 
org.apache.hadoop.hbase.HBaseConfiguration.checkDefaultsVersion(HBaseConfiguration.java:73)
at 
org.apache.hadoop.hbase.HBaseConfiguration.addHbaseResources(HBaseConfiguration.java:105)
at 
org.apache.hadoop.hbase.HBaseConfiguration.create(HBaseConfiguration.java:120)
at 
org.apache.hadoop.hbase.HBaseCommonTestingUtility.(HBaseCommonTestingUtility.java:46)
at 
org.apache.hadoop.hbase.TestClassFinder.(TestClassFinder.java:57)

Could not initialize class org.apache.hadoop.hbase.TestClassFinder
java.lang.NoClassDefFoundError: Could not initialize class 
org.apache.hadoop.hbase.TestClassFinder

Could not initialize class 
org.apache.hadoop.hbase.util.TestCoprocessorClassLoader
java.lang.NoClassDefFoundError: Could not initialize class 
org.apache.hadoop.hbase.util.TestCoprocessorClassLoader

Could not initialize class org.apache.hadoop.hbase.util.TestDynamicClassLoader
java.lang.NoClassDefFoundError: Could not initialize class 
org.apache.hadoop.hbase.util.TestDynamicClassLoader

*jdk=latest1.6,label=Hadoop*

java.lang.NoSuchFieldError: in
at 
org.apache.hadoop.hbase.codec.CellCodecWithTags$CellDecoder.parseCell(CellCodecWithTags.java:86)
at 
org.apache.hadoop.hbase.codec.BaseDecoder.advance(BaseDecoder.java:67)
at 
org.apache.hadoop.hbase.codec.TestCellCodecWithTags.testCellWithTag(TestCellCodecWithTags.java:76)

java.lang.NoSuchFieldError: in
at 
org.apache.hadoop.hbase.codec.KeyValueCodec$KeyValueDecoder.parseCell(KeyValueCodec.java:70)
at 
org.apache.hadoop.hbase.codec.BaseDecoder.advance(BaseDecoder.java:67)
at 
org.apache.hadoop.hbase.codec.TestKeyValueCodec.testOne(TestKeyValueCodec.java:80)



> Batch Deletes in MapReduce jobs (0.98)
> --
>
> Key: HBASE-14791
> URL: https://issues.apache.org/jira/browse/HBASE-14791
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.98.16
>Reporter: Lars Hofhansl
>Assignee: Alex Araujo
>  Labels: mapreduce
> Fix For: 0.98.17
>
> Attachments: HBASE-14791-0.98-v1.patch, HBASE-14791-0.98-v2.patch, 
> HBASE-14791-0.98.patch
>
>
> We found that some of our copy table job run for many hours, even when there 
> isn't that much data to copy.
> [~vik.karma] did his magic and found that the issue is with copying delete 
> markers (we use raw mode to also move deletes across).
> Looking at the code in 0.98 it's immediately obvious that deletes (unlike 
> puts) are not batched and hence sent to the other side one by one, causing a 
> network RTT for each delete marker.
> Looks like in trunk it's doing the right thing (using BufferedMutators for 
> all mutations in TableOutputFormat). So likely only a 0.98 (and 1.0, 1.1, 
> 1.2?) issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11393) Replication TableCfs should be a PB object rather than a string

[Heng Chen (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-11393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011131#comment-15011131
 ] 

Heng Chen commented on HBASE-11393:
---

Thanks [~ashish singhi] for your help to review. 
review board here.
https://reviews.apache.org/r/40443/

> Replication TableCfs should be a PB object rather than a string
> ---
>
> Key: HBASE-11393
> URL: https://issues.apache.org/jira/browse/HBASE-11393
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Enis Soztutar
> Fix For: 2.0.0
>
> Attachments: HBASE-11393.patch, HBASE-11393_v1.patch, 
> HBASE-11393_v2.patch, HBASE-11393_v3.patch, HBASE-11393_v4.patch, 
> HBASE-11393_v5.patch, HBASE-11393_v6.patch, HBASE-11393_v7.patch, 
> HBASE-11393_v8.patch, HBASE-11393_v9.patch
>
>
> We concatenate the list of tables and column families in format  
> "table1:cf1,cf2;table2:cfA,cfB" in zookeeper for table-cf to replication peer 
> mapping. 
> This results in ugly parsing code. We should do this a PB object. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011146#comment-15011146
 ] 

Sean Busbey commented on HBASE-14498:
-

severity impacts wether or not I will hold up the 1.2.0 release process for the 
issue. anything less than blocker means that release candidates won't wait.

> Master stuck in infinite loop when all Zookeeper servers are unreachable
> 
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
>  Issue Type: Bug
>  Components: master
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Blocker
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4
>
> Attachments: HBASE-14498-V2.patch, HBASE-14498-V3.patch, 
> HBASE-14498-V4.patch, HBASE-14498.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to 
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO 
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)] 
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in 
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and 
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO 
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)] 
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)] 
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN 
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)] 
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO 
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)] 
> zookeeper.ClientCnxn: Opening socket connection to server 
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown 
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)] 
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)] 
> zookeeper.ClientCnxn: Opening socket connection to server 
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown 
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)] 
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in 
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and 
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000] 
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, 
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181, 
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException: 
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)] 
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)] 
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)] 
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181. 
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt 
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an 
> > active master. 
> > HM2 is keep on waiting for region server to report him as part of active 
> > master intialization.
> {noformat} 
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting 
> for region servers count to settle; currently checked in 0, slept for 0 ms, 
> expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval 
> of 1500 ms. | 
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> ---
> ---
> 2015-09-26 15:32:50,841 | INFO | HM2-Host:21300.activeMasterManager | Waiting 
> for region servers count to settle; currently checked in 0, slept for 483913 
> ms, expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, 
> interval of 1500 ms. | 
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> {noformat}
> > At other end, region servers are reporting to HM1 on 3 sec interval. Here 
> > region server retrieve master location from zookeeper only when they 
> > couldn't connect to Master (ServiceException).
> Region Server will not report HM2 as per current design until unless HM1 
> abort,so HM2 will 

[jira] [Created] (HBASE-14835) isSplit is never set on HRegionInfo

[Elliott Clark (JIRA)]

Elliott Clark created HBASE-14835:
-

 Summary: isSplit is never set on HRegionInfo
 Key: HBASE-14835
 URL: https://issues.apache.org/jira/browse/HBASE-14835
 Project: HBase
  Issue Type: Bug
Reporter: Elliott Clark


Even if a region is hri.isSplit will always return false.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14782) FuzzyRowFilter skips valid rows

[Heng Chen (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15012495#comment-15012495
 ] 

Heng Chen commented on HBASE-14782:
---

If you insist on it, there is OK.  Let's just leave discussion here and fix 
this issue firstly.
+1 on patch v4.
I will push it later. 
Thanks [~vrodionov] for your nice patch.  
Thanks [~tedyu] and [~mahongbin] for your help to review.


> FuzzyRowFilter skips valid rows
> ---
>
> Key: HBASE-14782
> URL: https://issues.apache.org/jira/browse/HBASE-14782
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4, 0.98.17
>
> Attachments: HBASE-14782-0.98-v4.patch, HBASE-14782-v3.patch, 
> HBASE-14782-v4.patch, HBASE-14782.patch, HBASE-14782.patch
>
>
> The issue may affect not only master branch, but previous releases as well.
> This is from one of our customers:
> {quote}
> We are experiencing a problem with the FuzzyRowFilter for HBase scan. We 
> think that it is a bug. 
> Fuzzy filter should pick a row if it matches filter criteria irrespective of 
> other rows present in table but filter is dropping a row depending on some 
> other row present in table. 
> Details/Step to reproduce/Sample outputs below: 
> Missing row key: \x9C\x00\x044\x00\x00\x00\x00 
> Causing row key: \x9C\x00\x03\xE9e\xBB{X\x1Fwts\x1F\x15vRX 
> Prerequisites 
> 1. Create a test table. HBase shell command -- create 'fuzzytest','d' 
> 2. Insert some test data. HBase shell commands: 
> • put 'fuzzytest',"\x9C\x00\x044\x00\x00\x00\x00",'d:a','junk' 
> • put 'fuzzytest',"\x9C\x00\x044\x01\x00\x00\x00",'d:a','junk' 
> • put 'fuzzytest',"\x9C\x00\x044\x00\x01\x00\x00",'d:a','junk' 
> • put 'fuzzytest',"\x9C\x00\x044\x00\x00\x01\x00",'d:a','junk' 
> • put 'fuzzytest',"\x9C\x00\x044\x00\x01\x00\x01",'d:a','junk' 
> • put 'fuzzytest',"\x9B\x00\x044e\xBB\xB2\xBB",'d:a','junk' 
> • put 'fuzzytest',"\x9D\x00\x044e\xBB\xB2\xBB",'d:a','junk' 
> Now when you run the code, you will find \x9C\x00\x044\x00\x00\x00\x00 in 
> output because it matches filter criteria. (Refer how to run code below) 
> Insert the row key causing bug: 
> HBase shell command: put 
> 'fuzzytest',"\x9C\x00\x03\xE9e\xBB{X\x1Fwts\x1F\x15vRX",'d:a','junk' 
> Now when you run the code, you will not find \x9C\x00\x044\x00\x00\x00\x00 in 
> output even though it still matches filter criteria. 
> {quote}
> Verified the issue on master.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)