[jira] [Updated] (HBASE-22412) Improve the metrics in ByteBuffAllocator

2020-01-23 Thread Nick Dimiduk (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk updated HBASE-22412:
-
Fix Version/s: 2.3.0
   3.0.0

> Improve the metrics in ByteBuffAllocator
> 
>
> Key: HBASE-22412
> URL: https://issues.apache.org/jira/browse/HBASE-22412
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-22412.HBASE-21879.v1.patch, 
> HBASE-22412.HBASE-21879.v2.patch, HBASE-22412.HBASE-21879.v3.patch, JMX.png, 
> web-UI.png
>
>
> gAddress the comment in HBASE-22387: 
> bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
> the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for 
> this.
> Also I think we should use the allcated bytes instead of allocation number to 
> evaluate the heap allocation percent , so that we can decide whether the 
> ByteBuffer is too small and whether will have higher GC pressure.  Assume the 
> case:  the buffer size is 64KB, and each time we have a block with 65KB, then 
> it will have one heap allocation (1KB) and one pool allocation (64KB), if 
> only consider the allocation num, then the heap allocation ratio will be 1 / 
> (1 + 1) = 50%, but if consider the allocation bytes, the allocation ratio 
> will be  1KB / 65KB = 1.5%.
> If the heap allocation percent is less than  
> hbase.ipc.server.reservoir.minimal.allocating.size /  
> hbase.ipc.server.allocator.buffer.size,  then the allocator  works fine, 
> otherwise it's overload. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HBASE-22412) Improve the metrics in ByteBuffAllocator

2019-05-20 Thread Zheng Hu (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Hu updated HBASE-22412:
-
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

> Improve the metrics in ByteBuffAllocator
> 
>
> Key: HBASE-22412
> URL: https://issues.apache.org/jira/browse/HBASE-22412
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-22412.HBASE-21879.v1.patch, 
> HBASE-22412.HBASE-21879.v2.patch, HBASE-22412.HBASE-21879.v3.patch, JMX.png, 
> web-UI.png
>
>
> gAddress the comment in HBASE-22387: 
> bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
> the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for 
> this.
> Also I think we should use the allcated bytes instead of allocation number to 
> evaluate the heap allocation percent , so that we can decide whether the 
> ByteBuffer is too small and whether will have higher GC pressure.  Assume the 
> case:  the buffer size is 64KB, and each time we have a block with 65KB, then 
> it will have one heap allocation (1KB) and one pool allocation (64KB), if 
> only consider the allocation num, then the heap allocation ratio will be 1 / 
> (1 + 1) = 50%, but if consider the allocation bytes, the allocation ratio 
> will be  1KB / 65KB = 1.5%.
> If the heap allocation percent is less than  
> hbase.ipc.server.reservoir.minimal.allocating.size /  
> hbase.ipc.server.allocator.buffer.size,  then the allocator  works fine, 
> otherwise it's overload. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-22412) Improve the metrics in ByteBuffAllocator

2019-05-17 Thread Zheng Hu (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Hu updated HBASE-22412:
-
Description: 
gAddress the comment in HBASE-22387: 
bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for this.

Also I think we should use the allcated bytes instead of allocation number to 
evaluate the heap allocation percent , so that we can decide whether the 
ByteBuffer is too small and whether will have higher GC pressure.  Assume the 
case:  the buffer size is 64KB, and each time we have a block with 65KB, then 
it will have one heap allocation (1KB) and one pool allocation (64KB), if only 
consider the allocation num, then the heap allocation ratio will be 1 / (1 + 1) 
= 50%, but if consider the allocation bytes, the allocation ratio will be  1KB 
/ 65KB = 1.5%.

If the heap allocation percent is less than  
hbase.ipc.server.reservoir.minimal.allocating.size /  
hbase.ipc.server.allocator.buffer.size,  then the allocator  works fine, 
otherwise it's overload. 

  was:
Address the comment in HBASE-22387: 
bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for this.

Also I think we should use the allcated bytes instead of allocation number to 
evaluate the heap allocation percent , so that we can decide whether the 
ByteBuffer is too small and whether will have higher GC pressure.  Assume the 
case:  the buffer size is 64KB, and each time we have a block with 65KB, then 
it will have one heap allocation (1KB) and one pool allocation (64KB), if only 
consider the allocation num, then the heap allocation ratio will be 1 / (1 + 1) 
= 50%, but if consider the allocation bytes, the allocation ratio will be  1KB 
/ 65KB = 1.5%.

If the heap allocation percent is less than  
hbase.ipc.server.reservoir.minimal.allocating.size /  
hbase.ipc.server.allocator.buffer.size,  then the allocator  works fine, 
otherwise it's overload. 


> Improve the metrics in ByteBuffAllocator
> 
>
> Key: HBASE-22412
> URL: https://issues.apache.org/jira/browse/HBASE-22412
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-22412.HBASE-21879.v1.patch, 
> HBASE-22412.HBASE-21879.v2.patch, HBASE-22412.HBASE-21879.v3.patch, JMX.png, 
> web-UI.png
>
>
> gAddress the comment in HBASE-22387: 
> bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
> the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for 
> this.
> Also I think we should use the allcated bytes instead of allocation number to 
> evaluate the heap allocation percent , so that we can decide whether the 
> ByteBuffer is too small and whether will have higher GC pressure.  Assume the 
> case:  the buffer size is 64KB, and each time we have a block with 65KB, then 
> it will have one heap allocation (1KB) and one pool allocation (64KB), if 
> only consider the allocation num, then the heap allocation ratio will be 1 / 
> (1 + 1) = 50%, but if consider the allocation bytes, the allocation ratio 
> will be  1KB / 65KB = 1.5%.
> If the heap allocation percent is less than  
> hbase.ipc.server.reservoir.minimal.allocating.size /  
> hbase.ipc.server.allocator.buffer.size,  then the allocator  works fine, 
> otherwise it's overload. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-22412) Improve the metrics in ByteBuffAllocator

2019-05-16 Thread Zheng Hu (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Hu updated HBASE-22412:
-
Attachment: HBASE-22412.HBASE-21879.v3.patch

> Improve the metrics in ByteBuffAllocator
> 
>
> Key: HBASE-22412
> URL: https://issues.apache.org/jira/browse/HBASE-22412
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-22412.HBASE-21879.v1.patch, 
> HBASE-22412.HBASE-21879.v2.patch, HBASE-22412.HBASE-21879.v3.patch, JMX.png, 
> web-UI.png
>
>
> Address the comment in HBASE-22387: 
> bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
> the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for 
> this.
> Also I think we should use the allcated bytes instead of allocation number to 
> evaluate the heap allocation percent , so that we can decide whether the 
> ByteBuffer is too small and whether will have higher GC pressure.  Assume the 
> case:  the buffer size is 64KB, and each time we have a block with 65KB, then 
> it will have one heap allocation (1KB) and one pool allocation (64KB), if 
> only consider the allocation num, then the heap allocation ratio will be 1 / 
> (1 + 1) = 50%, but if consider the allocation bytes, the allocation ratio 
> will be  1KB / 65KB = 1.5%.
> If the heap allocation percent is less than  
> hbase.ipc.server.reservoir.minimal.allocating.size /  
> hbase.ipc.server.allocator.buffer.size,  then the allocator  works fine, 
> otherwise it's overload. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-22412) Improve the metrics in ByteBuffAllocator

2019-05-16 Thread Zheng Hu (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Hu updated HBASE-22412:
-
Attachment: JMX.png

> Improve the metrics in ByteBuffAllocator
> 
>
> Key: HBASE-22412
> URL: https://issues.apache.org/jira/browse/HBASE-22412
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-22412.HBASE-21879.v1.patch, 
> HBASE-22412.HBASE-21879.v2.patch, JMX.png, web-UI.png
>
>
> Address the comment in HBASE-22387: 
> bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
> the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for 
> this.
> Also I think we should use the allcated bytes instead of allocation number to 
> evaluate the heap allocation percent , so that we can decide whether the 
> ByteBuffer is too small and whether will have higher GC pressure.  Assume the 
> case:  the buffer size is 64KB, and each time we have a block with 65KB, then 
> it will have one heap allocation (1KB) and one pool allocation (64KB), if 
> only consider the allocation num, then the heap allocation ratio will be 1 / 
> (1 + 1) = 50%, but if consider the allocation bytes, the allocation ratio 
> will be  1KB / 65KB = 1.5%.
> If the heap allocation percent is less than  
> hbase.ipc.server.reservoir.minimal.allocating.size /  
> hbase.ipc.server.allocator.buffer.size,  then the allocator  works fine, 
> otherwise it's overload. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-22412) Improve the metrics in ByteBuffAllocator

2019-05-16 Thread Zheng Hu (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Hu updated HBASE-22412:
-
Attachment: web-UI.png

> Improve the metrics in ByteBuffAllocator
> 
>
> Key: HBASE-22412
> URL: https://issues.apache.org/jira/browse/HBASE-22412
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-22412.HBASE-21879.v1.patch, 
> HBASE-22412.HBASE-21879.v2.patch, web-UI.png
>
>
> Address the comment in HBASE-22387: 
> bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
> the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for 
> this.
> Also I think we should use the allcated bytes instead of allocation number to 
> evaluate the heap allocation percent , so that we can decide whether the 
> ByteBuffer is too small and whether will have higher GC pressure.  Assume the 
> case:  the buffer size is 64KB, and each time we have a block with 65KB, then 
> it will have one heap allocation (1KB) and one pool allocation (64KB), if 
> only consider the allocation num, then the heap allocation ratio will be 1 / 
> (1 + 1) = 50%, but if consider the allocation bytes, the allocation ratio 
> will be  1KB / 65KB = 1.5%.
> If the heap allocation percent is less than  
> hbase.ipc.server.reservoir.minimal.allocating.size /  
> hbase.ipc.server.allocator.buffer.size,  then the allocator  works fine, 
> otherwise it's overload. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-22412) Improve the metrics in ByteBuffAllocator

2019-05-15 Thread Zheng Hu (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Hu updated HBASE-22412:
-
Description: 
Address the comment in HBASE-22387: 
bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for this.

Also I think we should use the allcated bytes instead of allocation number to 
evaluate the heap allocation percent , so that we can decide whether the 
ByteBuffer is too small and whether will have higher GC pressure.  Assume the 
case:  the buffer size is 64KB, and each time we have a block with 65KB, then 
it will have one heap allocation (1KB) and one pool allocation (64KB), if only 
consider the allocation num, then the heap allocation ratio will be 1 / (1 + 1) 
= 50%, but if consider the allocation bytes, the allocation ratio will be  1KB 
/ 65KB = 1.5%.

If the heap allocation percent is less than  
hbase.ipc.server.reservoir.minimal.allocating.size /  
hbase.ipc.server.allocator.buffer.size,  then the allocator  works fine, 
otherwise it's overload. 

  was:
Address the comment in HBASE-22387: 
bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for this.

Also I think we should use the allcated bytes instead of allocation number to 
evaluate the heap allocation percent , so that we can decide whether the 
ByteBuffer is too small and whether will have higher GC pressure. 

If the heap allocation percent is less than  
hbase.ipc.server.reservoir.minimal.allocating.size /  
hbase.ipc.server.allocator.buffer.size,  then the allocator  works fine, 
otherwise it's overload. 


> Improve the metrics in ByteBuffAllocator
> 
>
> Key: HBASE-22412
> URL: https://issues.apache.org/jira/browse/HBASE-22412
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-22412.HBASE-21879.v1.patch, 
> HBASE-22412.HBASE-21879.v2.patch
>
>
> Address the comment in HBASE-22387: 
> bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
> the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for 
> this.
> Also I think we should use the allcated bytes instead of allocation number to 
> evaluate the heap allocation percent , so that we can decide whether the 
> ByteBuffer is too small and whether will have higher GC pressure.  Assume the 
> case:  the buffer size is 64KB, and each time we have a block with 65KB, then 
> it will have one heap allocation (1KB) and one pool allocation (64KB), if 
> only consider the allocation num, then the heap allocation ratio will be 1 / 
> (1 + 1) = 50%, but if consider the allocation bytes, the allocation ratio 
> will be  1KB / 65KB = 1.5%.
> If the heap allocation percent is less than  
> hbase.ipc.server.reservoir.minimal.allocating.size /  
> hbase.ipc.server.allocator.buffer.size,  then the allocator  works fine, 
> otherwise it's overload. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-22412) Improve the metrics in ByteBuffAllocator

2019-05-15 Thread Zheng Hu (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Hu updated HBASE-22412:
-
Attachment: HBASE-22412.HBASE-21879.v2.patch

> Improve the metrics in ByteBuffAllocator
> 
>
> Key: HBASE-22412
> URL: https://issues.apache.org/jira/browse/HBASE-22412
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-22412.HBASE-21879.v1.patch, 
> HBASE-22412.HBASE-21879.v2.patch
>
>
> Address the comment in HBASE-22387: 
> bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
> the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for 
> this.
> Also I think we should use the allcated bytes instead of allocation number to 
> evaluate the heap allocation percent , so that we can decide whether the 
> ByteBuffer is too small and whether will have higher GC pressure. 
> If the heap allocation percent is less than  
> hbase.ipc.server.reservoir.minimal.allocating.size /  
> hbase.ipc.server.allocator.buffer.size,  then the allocator  works fine, 
> otherwise it's overload. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-22412) Improve the metrics in ByteBuffAllocator

2019-05-14 Thread Zheng Hu (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Hu updated HBASE-22412:
-
Status: Patch Available  (was: Open)

> Improve the metrics in ByteBuffAllocator
> 
>
> Key: HBASE-22412
> URL: https://issues.apache.org/jira/browse/HBASE-22412
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-22412.HBASE-21879.v1.patch
>
>
> Address the comment in HBASE-22387: 
> bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
> the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for 
> this.
> Also I think we should use the allcated bytes instead of allocation number to 
> evaluate the heap allocation percent , so that we can decide whether the 
> ByteBuffer is too small and whether will have higher GC pressure. 
> If the heap allocation percent is less than  
> hbase.ipc.server.reservoir.minimal.allocating.size /  
> hbase.ipc.server.allocator.buffer.size,  then the allocator  works fine, 
> otherwise it's overload. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-22412) Improve the metrics in ByteBuffAllocator

2019-05-14 Thread Zheng Hu (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Hu updated HBASE-22412:
-
Attachment: HBASE-22412.HBASE-21879.v1.patch

> Improve the metrics in ByteBuffAllocator
> 
>
> Key: HBASE-22412
> URL: https://issues.apache.org/jira/browse/HBASE-22412
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-22412.HBASE-21879.v1.patch
>
>
> Address the comment in HBASE-22387: 
> bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
> the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for 
> this.
> Also I think we should use the allcated bytes instead of allocation number to 
> evaluate the heap allocation percent , so that we can decide whether the 
> ByteBuffer is too small and whether will have higher GC pressure. 
> If the heap allocation percent is less than  
> hbase.ipc.server.reservoir.minimal.allocating.size /  
> hbase.ipc.server.allocator.buffer.size,  then the allocator  works fine, 
> otherwise it's overload. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-22412) Improve the metrics in ByteBuffAllocator

2019-05-14 Thread Zheng Hu (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Hu updated HBASE-22412:
-
Description: 
Address the comment in HBASE-22387: 
bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for this.

Also I think we should use the allcated bytes instead of allocation number to 
evaluate the heap allocation percent , so that we can decide whether the 
ByteBuffer is too small and whether will have higher GC pressure. 

If the heap allocation percent is less than  
hbase.ipc.server.reservoir.minimal.allocating.size /  
hbase.ipc.server.allocator.buffer.size,  then the allocator  works fine, 
otherwise it's overload. 

  was:
Address the comment in HBASE-22387: 
bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for this.

Also I think we should use the allcated bytes instead of allocation number to 
evaluate the head allocation percent , so that we can decide whether the 
ByteBuffer is too small and will have higher GC pressure. 

If the heap allocation percent is less than  
hbase.ipc.server.reservoir.minimal.allocating.size /  
hbase.ipc.server.allocator.buffer.size,  then the allocator  works fine, 
otherwise it's overload. 


> Improve the metrics in ByteBuffAllocator
> 
>
> Key: HBASE-22412
> URL: https://issues.apache.org/jira/browse/HBASE-22412
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
>
> Address the comment in HBASE-22387: 
> bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
> the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for 
> this.
> Also I think we should use the allcated bytes instead of allocation number to 
> evaluate the heap allocation percent , so that we can decide whether the 
> ByteBuffer is too small and whether will have higher GC pressure. 
> If the heap allocation percent is less than  
> hbase.ipc.server.reservoir.minimal.allocating.size /  
> hbase.ipc.server.allocator.buffer.size,  then the allocator  works fine, 
> otherwise it's overload. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-22412) Improve the metrics in ByteBuffAllocator

2019-05-14 Thread Zheng Hu (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Hu updated HBASE-22412:
-
Description: 
Address the comment in HBASE-22387: 
bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for this.

Also I think we should use the allcated bytes instead of allocation number to 
evaluate the head allocation percent , so that we can decide whether the 
ByteBuffer is too small and will have higher GC pressure. 

If the heap allocation percent is less than  
hbase.ipc.server.reservoir.minimal.allocating.size /  
hbase.ipc.server.allocator.buffer.size,  then the allocator  works fine, 
otherwise it's overload. 

  was:
Address the comment in HBASE-22387: 
bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for this.

Also I think we should use the allcated bytes instead of allocation number to 
evaluate the head allocation percent , so that we can decide whether the 
ByteBuffer is too small and will have higher GC pressure. 


> Improve the metrics in ByteBuffAllocator
> 
>
> Key: HBASE-22412
> URL: https://issues.apache.org/jira/browse/HBASE-22412
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
>
> Address the comment in HBASE-22387: 
> bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
> the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for 
> this.
> Also I think we should use the allcated bytes instead of allocation number to 
> evaluate the head allocation percent , so that we can decide whether the 
> ByteBuffer is too small and will have higher GC pressure. 
> If the heap allocation percent is less than  
> hbase.ipc.server.reservoir.minimal.allocating.size /  
> hbase.ipc.server.allocator.buffer.size,  then the allocator  works fine, 
> otherwise it's overload. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-22412) Improve the metrics in ByteBuffAllocator

2019-05-14 Thread Zheng Hu (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Hu updated HBASE-22412:
-
Description: 
Address the comment in HBASE-22387: 
bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for this.

Also I think we should use the allcated bytes instead of allocation number to 
evaluate the head allocation percent , so that we can decide whether the 
ByteBuffer is too small and will have higher GC pressure. 

  was:
Address the comment in HBASE-22387: 
bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for this.


> Improve the metrics in ByteBuffAllocator
> 
>
> Key: HBASE-22412
> URL: https://issues.apache.org/jira/browse/HBASE-22412
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
>
> Address the comment in HBASE-22387: 
> bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
> the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for 
> this.
> Also I think we should use the allcated bytes instead of allocation number to 
> evaluate the head allocation percent , so that we can decide whether the 
> ByteBuffer is too small and will have higher GC pressure. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-22412) Improve the metrics in ByteBuffAllocator

2019-05-14 Thread Zheng Hu (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Hu updated HBASE-22412:
-
Summary: Improve the metrics in ByteBuffAllocator  (was: Replace the O(N) 
freeBufferCount by usedBufferCount in ByteBuffAllocator metrics)

> Improve the metrics in ByteBuffAllocator
> 
>
> Key: HBASE-22412
> URL: https://issues.apache.org/jira/browse/HBASE-22412
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
>
> Address the comment in HBASE-22387: 
> bq. The ByteBuffAllocator#getFreeBufferCount will be O(N) complexity, because 
> the buffers here is an ConcurrentLinkedQueue. It's worth file an issue for 
> this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)