[jira] [Commented] (HBASE-17788) Procedure V2 performance improvements
[ https://issues.apache.org/jira/browse/HBASE-17788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16397765#comment-16397765 ] stack commented on HBASE-17788: --- Unscheduled from 2.0.0. Nice to have and we'll need it when we want to scale. > Procedure V2 performance improvements > - > > Key: HBASE-17788 > URL: https://issues.apache.org/jira/browse/HBASE-17788 > Project: HBase > Issue Type: Improvement > Components: amv2, Performance, proc-v2 >Reporter: stack >Priority: Major > > This is a list of items to work on to improve framework perf taken from > https://docs.google.com/document/d/1kEWzyA0iCyRjdogjju9JNMDT9ODHaKcyoAJTMXkDpxc/edit# > Make sub-issues to work on each in the list below: > * Replace fixed Executor Threads with dynamic thread pool (Easy) > * Connect the active thread count with the ProcStore slots (Medium) > * Parallelize the load (Complex) > * Allow Procedures to start early (Complex) > https://github.com/apache/hbase/blob/master/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/ProcedureWALFormatReader.java#L155 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-17788) Procedure V2 performance improvements
[ https://issues.apache.org/jira/browse/HBASE-17788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943046#comment-15943046 ] Janos Gub commented on HBASE-17788: --- Converted to issue to be able to create subtasks. Issue https://issues.apache.org/jira/browse/HBASE-12439 is the parent of this one. > Procedure V2 performance improvements > - > > Key: HBASE-17788 > URL: https://issues.apache.org/jira/browse/HBASE-17788 > Project: HBase > Issue Type: Improvement > Components: Performance, proc-v2 >Reporter: stack > Fix For: 2.0.0 > > > This is a list of items to work on to improve framework perf taken from > https://docs.google.com/document/d/1kEWzyA0iCyRjdogjju9JNMDT9ODHaKcyoAJTMXkDpxc/edit# > Make sub-issues to work on each in the list below: > * Replace fixed Executor Threads with dynamic thread pool (Easy) > * Connect the active thread count with the ProcStore slots (Medium) > * Parallelize the load (Complex) > * Allow Procedures to start early (Complex) > https://github.com/apache/hbase/blob/master/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/ProcedureWALFormatReader.java#L155 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17788) Procedure V2 performance improvements
[ https://issues.apache.org/jira/browse/HBASE-17788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940783#comment-15940783 ] stack commented on HBASE-17788: --- Fantastic [~gubjanos] Thanks for the help. Make yourself an issue and link it here. I added you as a contributor so you should be able to hack around in JIRA. bq. Just to clarify, is this about the WorkerThreads in ProcedureExecutor.java? Because the construction there is somewhat similar to a ThreadPool Executor. That is my understanding. They run the procedures. What you thinking? Replace w/ ThreadPool Executor? That sounds good. There is means of monitoring for stuck threads/procedures. We need that. Reporting on what is stuck could be better; IIRC, it just says 'stuck' w/o saying which procedure is stopped. Thanks again. > Procedure V2 performance improvements > - > > Key: HBASE-17788 > URL: https://issues.apache.org/jira/browse/HBASE-17788 > Project: HBase > Issue Type: Sub-task > Components: Performance, proc-v2 >Reporter: stack > Fix For: 2.0.0 > > > This is a list of items to work on to improve framework perf taken from > https://docs.google.com/document/d/1kEWzyA0iCyRjdogjju9JNMDT9ODHaKcyoAJTMXkDpxc/edit# > Make sub-issues to work on each in the list below: > * Replace fixed Executor Threads with dynamic thread pool (Easy) > * Connect the active thread count with the ProcStore slots (Medium) > * Parallelize the load (Complex) > * Allow Procedures to start early (Complex) > https://github.com/apache/hbase/blob/master/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/ProcedureWALFormatReader.java#L155 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17788) Procedure V2 performance improvements
[ https://issues.apache.org/jira/browse/HBASE-17788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940235#comment-15940235 ] Janos Gub commented on HBASE-17788: --- bq. Replace fixed Executor Threads with dynamic thread pool (Easy) I was starting to work on this one. Just to clarify, is this about the WorkerThreads in ProcedureExecutor.java? Because the construction there is somewhat similar to a ThreadPool Executor. > Procedure V2 performance improvements > - > > Key: HBASE-17788 > URL: https://issues.apache.org/jira/browse/HBASE-17788 > Project: HBase > Issue Type: Sub-task > Components: Performance, proc-v2 >Reporter: stack > Fix For: 2.0.0 > > > This is a list of items to work on to improve framework perf taken from > https://docs.google.com/document/d/1kEWzyA0iCyRjdogjju9JNMDT9ODHaKcyoAJTMXkDpxc/edit# > Make sub-issues to work on each in the list below: > * Replace fixed Executor Threads with dynamic thread pool (Easy) > * Connect the active thread count with the ProcStore slots (Medium) > * Parallelize the load (Complex) > * Allow Procedures to start early (Complex) > https://github.com/apache/hbase/blob/master/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/ProcedureWALFormatReader.java#L155 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
