[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913320#comment-16913320 ] Hudson commented on HBASE-22810: Results for branch branch-2 [build #2184 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2184/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2184//General_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2184//JDK8_Nightly_Build_Report_(Hadoop2)/] (/) {color:green}+1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2184//JDK8_Nightly_Build_Report_(Hadoop3)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (x) {color:red}-1 client integration test{color} --Failed when running client tests on top of Hadoop 3. [see log for details|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2184//artifact/output-integration/hadoop-3.log]. (note that this means we didn't check the Hadoop 3 shaded client) > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913291#comment-16913291 ] Hudson commented on HBASE-22810: Results for branch master [build #1352 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1352/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/master/1352//General_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/master/1352//JDK8_Nightly_Build_Report_(Hadoop2)/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://builds.apache.org/job/HBase%20Nightly/job/master/1352//JDK8_Nightly_Build_Report_(Hadoop3)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913226#comment-16913226 ] Hudson commented on HBASE-22810: Results for branch branch-2.1 [build #1509 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1509/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1509//General_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1509//JDK8_Nightly_Build_Report_(Hadoop2)/] (/) {color:green}+1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1509//JDK8_Nightly_Build_Report_(Hadoop3)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913223#comment-16913223 ] Hudson commented on HBASE-22810: Results for branch branch-2.2 [build #533 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/533/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/533//General_Nightly_Build_Report/] (/) {color:green}+1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/533//JDK8_Nightly_Build_Report_(Hadoop2)/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/533//JDK8_Nightly_Build_Report_(Hadoop3)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913211#comment-16913211 ] Hudson commented on HBASE-22810: Results for branch branch-1 [build #1021 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1021/]: (x) *{color:red}-1 overall{color}* details (if available): (x) {color:red}-1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1021//General_Nightly_Build_Report/] (x) {color:red}-1 jdk7 checks{color} -- For more information [see jdk7 report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1021//JDK7_Nightly_Build_Report/] (/) {color:green}+1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1021//JDK8_Nightly_Build_Report_(Hadoop2)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913207#comment-16913207 ] Hudson commented on HBASE-22810: Results for branch branch-1.4 [build #971 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/971/]: (x) *{color:red}-1 overall{color}* details (if available): (x) {color:red}-1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/971//General_Nightly_Build_Report/] (x) {color:red}-1 jdk7 checks{color} -- For more information [see jdk7 report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/971//JDK7_Nightly_Build_Report/] (/) {color:green}+1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/971//JDK8_Nightly_Build_Report_(Hadoop2)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913202#comment-16913202 ] Hudson commented on HBASE-22810: Results for branch branch-1.3 [build #931 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/931/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/931//General_Nightly_Build_Report/] (/) {color:green}+1 jdk7 checks{color} -- For more information [see jdk7 report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/931//JDK7_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/931//JDK8_Nightly_Build_Report_(Hadoop2)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913042#comment-16913042 ] Hudson commented on HBASE-22810: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #623 (See [https://builds.apache.org/job/HBase-1.3-IT/623/]) HBASE-22810 Initialize an separate ThreadPoolExecutor for (openinx: rev 40cf771a825cf657f5f7cb0f7ef8c986e5b1fe4c) * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestConcurrentFlushSnapshotFromClient.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/EnabledTableSnapshotHandler.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestFlushSnapshotFromClient.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16912993#comment-16912993 ] Zheng Hu commented on HBASE-22810: -- Committed the addendum to all branches, Thanks all. > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11, 2.0.7 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16912278#comment-16912278 ] Zheng Hu commented on HBASE-22810: -- Created a PR to address the above uniform & add a UT. > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11, 2.0.7 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16912256#comment-16912256 ] Zheng Hu commented on HBASE-22810: -- Thanks [~stack] for the fix. Read the UT code, it's indeed a test which easy to be flaky. For example, all snapshot request are submitted but the snapshot is a bit slow, none are completed when the assert begin: {code} +assertTrue("We expect at least 1 request to be rejected because of we concurrently" + +" issued many requests", takenSize < ssNum && takenSize > 0); {code} Then, the assert will be failure. so +1 for me to remove it (I guess after increasing the 'hbase.master.executor.snapshot.threads', it's easy to happen now). [~an...@apache.org], Thanks for the reminding . It's true, there are two different config keys for the snapshot threads size, but I think they have different meanings: 1. hbase.master.executor.snapshot.threads : means how many snapshot requests from client we can handle at master side the same time; 2. hbase.snapshot.master.threads: how many snapshot procedure we can coordinator with region server. The config key#1 limit the all the snapshot request, while the key#2 only limit the snapshot procedure with RS ( it's a part of the snapshot request).Maybe we can uniform the two config keys into one ? although we will initialize two different thread pools with the same thread size for different purpose. > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11, 2.0.7 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911898#comment-16911898 ] Ankit Singhal commented on HBASE-22810: --- [~openinx], It seems SnapshotManager configures pools separately which allow only 1 thread and that's why TestFlushSnapshotFromClient#testConcurrentSnapshottingAttempts is failing, so you may also need to update the below code with your new config to keep the threads consistent. SnapshotManager#initialize {code} int opThreads = conf.getInt(SNAPSHOT_POOL_THREADS_KEY, SNAPSHOT_POOL_THREADS_DEFAULT); {code} to {code} int opThreads = conf.getInt(MASTER_SNAPSHOT_OPERATIONS_THREADS, MASTER_SNAPSHOT_OPERATIONS_THREADS_DEFAULT); {code} And, committing improvements which change the defaults value in patch releases(2.[0-2].x), doesn't it affect our operational compatibility? > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11, 2.0.7 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16910973#comment-16910973 ] stack commented on HBASE-22810: --- Ok. This did not make it into 2.0.6RC1 which just became 2.0.6. I moved fix version to 2.0.7 though there will be no more releases on branch-2.0. I just saw this issue and how it breaks TestFlushSnapshotFromClient. I looked at the TestFlushSnapshotFromClient failures independently and to me the test looked flakey anyways and was seemingly written flakey in the first place so just removed it with the below commit. Did not go back to branch-1. After seeing this issue, I was a mite reactionary it seems. {code} commit 05b88af057d35e247bf37b725f2735292f944387 (HEAD -> 2.0, origin/branch-2.0) Author: stack Date: Mon Aug 19 14:25:21 2019 -0700 HBASE-22882 TestFlushSnapshotFromClient#testConcurrentSnapshottingAttempts is flakey (was written flakey) Addendum; just remove the test altogether {code} > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11, 2.0.7 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16909963#comment-16909963 ] Zheng Hu commented on HBASE-22810: -- Fine. Sorry , I think I forgot this issue before, Let me dig this . > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16909956#comment-16909956 ] Duo Zhang commented on HBASE-22810: --- OK, seems it also breaks the tests on branch-2.1... https://builds.apache.org/job/HBASE-Find-Flaky-Tests/job/branch-2.1/lastSuccessfulBuild/artifact/dashboard.html Could you please quickly identify whether this is a test issue or not? If not, we need to sink all the pending RCs and prepare new ones. Thanks. [~openinx] > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908631#comment-16908631 ] Duo Zhang commented on HBASE-22810: --- It also breaks the test on master. > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908626#comment-16908626 ] Zheng Hu commented on HBASE-22810: -- OK, Thanks [~apurtell], Let me take a look, will provide the addendum soon if necessary. > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908554#comment-16908554 ] Hudson commented on HBASE-22810: Results for branch branch-2.2 [build #517 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/517/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/517//General_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/517//JDK8_Nightly_Build_Report_(Hadoop2)/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/517//JDK8_Nightly_Build_Report_(Hadoop3)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (x) {color:red}-1 client integration test{color} --Failed when running client tests on top of Hadoop 2. [see log for details|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/517//artifact/output-integration/hadoop-2.log]. (note that this means we didn't run on Hadoop 3) > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908431#comment-16908431 ] Hudson commented on HBASE-22810: Results for branch branch-2 [build #2169 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2169/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2169//General_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2169//JDK8_Nightly_Build_Report_(Hadoop2)/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2169//JDK8_Nightly_Build_Report_(Hadoop3)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908347#comment-16908347 ] Hudson commented on HBASE-22810: Results for branch branch-1.3 [build #920 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/920/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/920//General_Nightly_Build_Report/] (x) {color:red}-1 jdk7 checks{color} -- For more information [see jdk7 report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/920//JDK7_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/920//JDK8_Nightly_Build_Report_(Hadoop2)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908345#comment-16908345 ] Hudson commented on HBASE-22810: Results for branch branch-1.4 [build #959 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/959/]: (x) *{color:red}-1 overall{color}* details (if available): (x) {color:red}-1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/959//General_Nightly_Build_Report/] (x) {color:red}-1 jdk7 checks{color} -- For more information [see jdk7 report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/959//JDK7_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/959//JDK8_Nightly_Build_Report_(Hadoop2)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908234#comment-16908234 ] Hudson commented on HBASE-22810: Results for branch branch-2.0 [build #1867 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1867/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1867//General_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1867//JDK8_Nightly_Build_Report_(Hadoop2)/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1867//JDK8_Nightly_Build_Report_(Hadoop3)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908225#comment-16908225 ] Hudson commented on HBASE-22810: Results for branch branch-2.1 [build #1486 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1486/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1486//General_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1486//JDK8_Nightly_Build_Report_(Hadoop2)/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1486//JDK8_Nightly_Build_Report_(Hadoop3)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908058#comment-16908058 ] Hudson commented on HBASE-22810: Results for branch master [build #1335 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1335/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/master/1335//General_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/master/1335//JDK8_Nightly_Build_Report_(Hadoop2)/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://builds.apache.org/job/HBase%20Nightly/job/master/1335//JDK8_Nightly_Build_Report_(Hadoop3)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907988#comment-16907988 ] Hudson commented on HBASE-22810: Results for branch branch-1 [build #1009 on builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1009/]: (x) *{color:red}-1 overall{color}* details (if available): (x) {color:red}-1 general checks{color} -- For more information [see general report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1009//General_Nightly_Build_Report/] (x) {color:red}-1 jdk7 checks{color} -- For more information [see jdk7 report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1009//JDK7_Nightly_Build_Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1009//JDK8_Nightly_Build_Report_(Hadoop2)/] (/) {color:green}+1 source release artifact{color} -- See build output for details. > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22810) Initialize an separate ThreadPoolExecutor for taking/restoring snapshot
[ https://issues.apache.org/jira/browse/HBASE-22810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907867#comment-16907867 ] Hudson commented on HBASE-22810: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #618 (See [https://builds.apache.org/job/HBase-1.3-IT/618/]) HBASE-22810 Initialize an separate ThreadPoolExecutor for (openinx: rev b97621b30216dc39ad605e06738a070a69a47da3) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/executor/ExecutorType.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/executor/EventType.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/executor/TestExecutorService.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java > Initialize an separate ThreadPoolExecutor for taking/restoring snapshot > > > Key: HBASE-22810 > URL: https://issues.apache.org/jira/browse/HBASE-22810 > Project: HBase > Issue Type: Improvement >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > In EventType class, we have the following definition, means taking snapshot > & restoring snapshot are use the MASTER_TABLE_OPERATIONS Executor now. > {code} > /** >* Messages originating from Client to Master. >* C_M_SNAPSHOT_TABLE >* Client asking Master to snapshot an offline table. >*/ > C_M_SNAPSHOT_TABLE(48, ExecutorType.MASTER_TABLE_OPERATIONS), > /** >* Messages originating from Client to Master. >* C_M_RESTORE_SNAPSHOT >* Client asking Master to restore a snapshot. >*/ > C_M_RESTORE_SNAPSHOT (49, ExecutorType.MASTER_TABLE_OPERATIONS), > {code} > But when I checked the MASTER_TABLE_OPERATIONS thread pool initialization, I > see : > {code} > private void startServiceThreads() throws IOException{ >// ... some other code initializing >// We depend on there being only one instance of this executor running >// at a time. To do concurrency, would need fencing of enable/disable of >// tables. >// Any time changing this maxThreads to > 1, pls see the comment at >// AccessController#postCompletedCreateTableAction > > this.executorService.startExecutorService(ExecutorType.MASTER_TABLE_OPERATIONS, > 1); >startProcedureExecutor(); > {code} > That's to say, for CPs enable or disable table sequencely, we will create > a ThreadPoolExecutor with threadPoolSize=1. Then we actually cann't > accomplish the snapshoting concurrence even if they are total difference > tables, says if there are two table snapshoting request, and the Table A cost > 5min for snapshoting, then the Table B need to wait 5min and once Table A > finish its snapshot , then Table B will start the snapshot. > While we've setting the snapshot timeout, so it will be easy to timeout for > table B snapshoting . Actually, we can create a separate thead pool for > snapshot operations only. -- This message was sent by Atlassian JIRA (v7.6.14#76016)