[jira] [Commented] (HBASE-3909) Add dynamic config
[ https://issues.apache.org/jira/browse/HBASE-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13248459#comment-13248459 ] Subbu M Iyer commented on HBASE-3909: - @Uma: 5. getMaster().updateConfig is not deprecated. It is used by the Shell. 7. If ZK is down or connectivity is lost we have much bigger problem then loosing a config update. 8. Yes. I thought of creating two buckets of configuration elements. 1) Bootstrap config items which may not be changed after the cluster is up or have no effect changing them once cluster is up 2) Non bootstrap, truly runtime configurations which have immediate effect. My thought was to add these features if there is a consensus from the group. 9. Yes. But if a user is changing the configuration either in XML or dynamic he is better aware of the implications. no? 10. NodeDeleted is not required. We are not going to delete a configuration entry at runtime and even doing so may not have any effect now. In the future if we want to trigger some actions based on presence/absence of some configuration entries this might be added. 11. Yes, we can offer a Master/RS specific fine grained configuration controls if that is a real requirement. My thought is begin with some thing very simple and add complexities as we see fit. So, one option is as we all know is to create separate subset of config nodes per Master/RS in the cluster and they subscribe to events specific to their own subset. But, if you look at our current configuration xml we make no clear distinction as to which properties belong to which runtime entity. Although we name them like config.key.master or config.key.region, I am not sure whether we can count on them to make a decision on whether a property applies to MAster only or RS only and so on. 12. My thought is InMemory Configuration holds the config items after creating a ZK node for the same as other wise it may not be altered. Basically we need to make sure that ZK config node exists for a config key before we keep it in memory. 13. My thought is ideally we want Master to be the single point of source for all cluster wide config management/updates. We definitely don't want to directly manipulate a RS specific property. I mean we should not expose API's in RS to enable config key manipulations. Master should be the single point of source for all configuration entries irrespective of whether the config is cluster wide, master specific or RS specific and ZK should be the medium of communication across the cluster. So, master will be involved in all config updates is my take. Others please comment. Add dynamic config -- Key: HBASE-3909 URL: https://issues.apache.org/jira/browse/HBASE-3909 Project: HBase Issue Type: Bug Reporter: stack Fix For: 0.96.0 Attachments: 3909-v1.patch, 3909.v1 I'm sure this issue exists already, at least as part of the discussion around making online schema edits possible, but no hard this having its own issue. Ted started a conversation on this topic up on dev and Todd suggested we lookd at how Hadoop did it over in HADOOP-7001 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5715) Revert 'Instant schema alter' for now, HBASE-4213
[ https://issues.apache.org/jira/browse/HBASE-5715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247347#comment-13247347 ] Subbu M Iyer commented on HBASE-5715: - Lars: I have two issues to address (Please add more if I missed any) 1. MonitoredTask usage is incorrect and needs a revisit. 2. Introduce throttling during the Region open/close requests. Given my current schedules i would imagine that this will be couple of days of effort. I would like to understand how are we going to address like track it as part of original Jira or create new one? Patches will be based on trunk or branch et al. Revert 'Instant schema alter' for now, HBASE-4213 - Key: HBASE-5715 URL: https://issues.apache.org/jira/browse/HBASE-5715 Project: HBase Issue Type: Task Reporter: stack Attachments: revert.txt, revert.v2.txt, revert.v3.txt, revert.v4.txt See this discussion: http://search-hadoop.com/m/NxCQh1KlSxR1/Pull+instant+schema+updating+out%253Fsubj=Pull+instant+schema+updating+out+ Pull out hbase-4213 for now. Can add it back later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5715) Revert 'Instant schema alter' for now, HBASE-4213
[ https://issues.apache.org/jira/browse/HBASE-5715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13247349#comment-13247349 ] Subbu M Iyer commented on HBASE-5715: - Stack: Could you please help me with a branch for this as this definitely needs some baking time before we feel comfortable on a suitable release branch ? Revert 'Instant schema alter' for now, HBASE-4213 - Key: HBASE-5715 URL: https://issues.apache.org/jira/browse/HBASE-5715 Project: HBase Issue Type: Task Reporter: stack Attachments: revert.txt, revert.v2.txt, revert.v3.txt, revert.v4.txt See this discussion: http://search-hadoop.com/m/NxCQh1KlSxR1/Pull+instant+schema+updating+out%253Fsubj=Pull+instant+schema+updating+out+ Pull out hbase-4213 for now. Can add it back later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5715) Revert 'Instant schema alter' for now, HBASE-4213
[ https://issues.apache.org/jira/browse/HBASE-5715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13248035#comment-13248035 ] Subbu M Iyer commented on HBASE-5715: - @Lars Thanks for the helping hands. Do you want to address the open/close throttling while I redo the monitoring part? Let me know. @Ted: Thanks for your help with the testing. Will post my patch once I am done to this Jira. Revert 'Instant schema alter' for now, HBASE-4213 - Key: HBASE-5715 URL: https://issues.apache.org/jira/browse/HBASE-5715 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Fix For: 0.94.0 Attachments: revert.txt, revert.v2.txt, revert.v3.txt, revert.v4.txt, revert094.v4.txt See this discussion: http://search-hadoop.com/m/NxCQh1KlSxR1/Pull+instant+schema+updating+out%253Fsubj=Pull+instant+schema+updating+out+ Pull out hbase-4213 for now. Can add it back later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3909) Add dynamic config
[ https://issues.apache.org/jira/browse/HBASE-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13248041#comment-13248041 ] Subbu M Iyer commented on HBASE-3909: - @Uma/@Ted: Thanks a lot for your review and comments. As I mentioned this is early draft and will address your concerns in the next few days. Also @Uma you raised lot of interesting questions which I had while working on this and I will post my thoughts on the same some time tomorrow. thanks Add dynamic config -- Key: HBASE-3909 URL: https://issues.apache.org/jira/browse/HBASE-3909 Project: HBase Issue Type: Bug Reporter: stack Fix For: 0.96.0 Attachments: 3909-v1.patch, 3909.v1 I'm sure this issue exists already, at least as part of the discussion around making online schema edits possible, but no hard this having its own issue. Ted started a conversation on this topic up on dev and Todd suggested we lookd at how Hadoop did it over in HADOOP-7001 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5715) Revert 'Instant schema alter' for now, HBASE-4213
[ https://issues.apache.org/jira/browse/HBASE-5715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13246490#comment-13246490 ] Subbu M Iyer commented on HBASE-5715: - I reviewed the thread above and I do see mixed feeling about this. So, just wanted to clarify whether addressing the monitoring leaks (5702), introduce throttling to address rapid open/close region concerns (5713) and addressing the ambiguity in the flags that enables/disables this feature is even an option or not? I can take up addressing the above two minor issues if that's an option. Also, since the default alter does not go through this path the impact to the common user should be none. Please note that what we have is based on loose set of requirements and I sincerely think that what is lacking is rock solid testing in a production kind-of environment and having cycles to address the issues if any. Revert 'Instant schema alter' for now, HBASE-4213 - Key: HBASE-5715 URL: https://issues.apache.org/jira/browse/HBASE-5715 Project: HBase Issue Type: Task Reporter: stack See this discussion: http://search-hadoop.com/m/NxCQh1KlSxR1/Pull+instant+schema+updating+out%253Fsubj=Pull+instant+schema+updating+out+ Pull out hbase-4213 for now. Can add it back later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3909) Add dynamic config
[ https://issues.apache.org/jira/browse/HBASE-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13246828#comment-13246828 ] Subbu M Iyer commented on HBASE-3909: - Ted, I am trying to but for some reason RB is not allowing me to attach my svn patch. Is there anything special i need to do? I always have issues attaching my patches to RB. will keep trying. thanks Subbu On Wed, Apr 4, 2012 at 4:08 PM, Zhihong Yu (Commented) (JIRA) Add dynamic config -- Key: HBASE-3909 URL: https://issues.apache.org/jira/browse/HBASE-3909 Project: HBase Issue Type: Bug Reporter: stack Fix For: 0.96.0 Attachments: 3909-v1.patch I'm sure this issue exists already, at least as part of the discussion around making online schema edits possible, but no hard this having its own issue. Ted started a conversation on this topic up on dev and Todd suggested we lookd at how Hadoop did it over in HADOOP-7001 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5702) MasterSchemaChangeTracker.excludeRegionServerForSchemaChanges leaks a MonitoredTask per call
[ https://issues.apache.org/jira/browse/HBASE-5702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13245650#comment-13245650 ] Subbu M Iyer commented on HBASE-5702: - JD, Can you share with me the many more issues with MonitoredTask that you have mentioned? I believe the problems you have seen are all related to the MonitoredTask reporting during instant schema change process and not with the actual schema change process itself? Please let me know so we can categorize and address the issues accordingly. MasterSchemaChangeTracker.excludeRegionServerForSchemaChanges leaks a MonitoredTask per call Key: HBASE-5702 URL: https://issues.apache.org/jira/browse/HBASE-5702 Project: HBase Issue Type: Bug Affects Versions: 0.94.0 Reporter: Jean-Daniel Cryans Assignee: Subbu M Iyer Priority: Critical Fix For: 0.94.0, 0.96.0 This bug is so easy to reproduce I'm wondering why it hasn't been reported yet. Stop any number of region servers on a 0.94/6 cluster and you'll see in the master interface one task per stopped region server saying the following: |Processing schema change exclusion for region server = sv4r27s44,62023,1333402175340|RUNNING (since 5sec ago)|No schema change in progress. Skipping exclusion for server = sv4r27s44,62023,1333402175340 (since 5sec ago)| It's gonna stay there until the master cleans it: bq. WARN org.apache.hadoop.hbase.monitoring.TaskMonitor: Status Processing schema change exclusion for region server = sv4r27s44,62023,1333402175340: status=No schema change in progress. Skipping exclusion for server = sv4r27s44,62023,1333402175340, state=RUNNING, startTime=1333404636419, completionTime=-1 appears to have been leaked It's not clear to me why it's using a MonitoredTask in the first place. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13145961#comment-13145961 ] Subbu M Iyer commented on HBASE-4213: - - New patch to address concurrent schema alter and region splits. - Added two new tests to test the concurrent split + alter scenarios. Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK. Key: HBASE-4213 URL: https://issues.apache.org/jira/browse/HBASE-4213 Project: HBase Issue Type: Improvement Reporter: Subbu M Iyer Assignee: Subbu M Iyer Fix For: 0.92.0 Attachments: 4213-0.92.txt, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Fixed_NPE_in_RS_during_alter_.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-Nov072011-Patch_to_support_concurrent_split_and_alter__.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch This Jira is a slight variation in approach to what is being done as part of https://issues.apache.org/jira/browse/HBASE-1730 Support instant schema updates such as Modify Table, Add Column, Modify Column operations: 1. With out enable/disabling the table. 2. With out bulk unassign/assign of regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13144773#comment-13144773 ] Subbu M Iyer commented on HBASE-4213: - JD, Fixed the NPE that is causing the RS to die. All tests pass and don't see this exception. Would be great if you can give it one more try with your concurrent alter + split scenario. I am also adding a test case for the concurrent split + alter scenario and I am validating the same. Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK. Key: HBASE-4213 URL: https://issues.apache.org/jira/browse/HBASE-4213 Project: HBase Issue Type: Improvement Reporter: Subbu M Iyer Assignee: Subbu M Iyer Fix For: 0.92.0 Attachments: 4213-0.92.txt, 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-102511.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-Nov-2-2011_patch_.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch This Jira is a slight variation in approach to what is being done as part of https://issues.apache.org/jira/browse/HBASE-1730 Support instant schema updates such as Modify Table, Add Column, Modify Column operations: 1. With out enable/disabling the table. 2. With out bulk unassign/assign of regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13126326#comment-13126326 ] Subbu M Iyer commented on HBASE-4213: - Latest patch supports: 1. LB aware schema change process. 2. Monitoring support to show the alter status/error etc. 3. Progress reporting. 4. Added extensive test scenarios to test various failover/failure scenarios. (17 test cases so far) Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK. Key: HBASE-4213 URL: https://issues.apache.org/jira/browse/HBASE-4213 Project: HBase Issue Type: Improvement Reporter: Subbu M Iyer Assignee: Subbu M Iyer Fix For: 0.92.0 Attachments: 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch This Jira is a slight variation in approach to what is being done as part of https://issues.apache.org/jira/browse/HBASE-1730 Support instant schema updates such as Modify Table, Add Column, Modify Column operations: 1. With out enable/disabling the table. 2. With out bulk unassign/assign of regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13126333#comment-13126333 ] Subbu M Iyer commented on HBASE-4213: - Ted, Yes, I will. My trunk is from 09/12. I wanted to get some initial feedback. This time I could successfully request for a RB request. :-):-) Let me know how we handle this LB integration makes sense to you. thanks Subbu Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK. Key: HBASE-4213 URL: https://issues.apache.org/jira/browse/HBASE-4213 Project: HBase Issue Type: Improvement Reporter: Subbu M Iyer Assignee: Subbu M Iyer Fix For: 0.92.0 Attachments: 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch This Jira is a slight variation in approach to what is being done as part of https://issues.apache.org/jira/browse/HBASE-1730 Support instant schema updates such as Modify Table, Add Column, Modify Column operations: 1. With out enable/disabling the table. 2. With out bulk unassign/assign of regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4213) Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK.
[ https://issues.apache.org/jira/browse/HBASE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13126348#comment-13126348 ] Subbu M Iyer commented on HBASE-4213: - Thanks for your support on this Ted. Also, on my local (with only one RS), for a 70 region test table I see a more than 100+% improvement in alter performance. For a table in a bigger cluster with large number of regions, we should see drastic difference in performance. thanks Subbu Support for fault tolerant, instant schema updates with out master's intervention (i.e with out enable/disable and bulk assign/unassign) through ZK. Key: HBASE-4213 URL: https://issues.apache.org/jira/browse/HBASE-4213 Project: HBase Issue Type: Improvement Reporter: Subbu M Iyer Assignee: Subbu M Iyer Fix For: 0.92.0 Attachments: 4213-101211-Support_instant_schema_changes_through_ZK.patch, 4213-Instant_Schema_change_through_ZK.patch, 4213-V10-Support_instant_schema_changes_through_ZK.patch, 4213-V5-Support_instant_schema_changes_through_ZK.patch, 4213-V7-Support_instant_schema_changes_through_ZK.patch, 4213-V8-Support_instant_schema_changes_through_ZK.patch, 4213-V9-Support_instant_schema_changes_through_ZK.patch, 4213-v9.txt, 4213.v6, HBASE-4213-Instant_schema_change.patch, HBASE-4213_Instant_schema_change_-Version_2_.patch, HBASE_Instant_schema_change-version_3_.patch This Jira is a slight variation in approach to what is being done as part of https://issues.apache.org/jira/browse/HBASE-1730 Support instant schema updates such as Modify Table, Add Column, Modify Column operations: 1. With out enable/disabling the table. 2. With out bulk unassign/assign of regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
