Re: Review Request 28976: Rolling Upgrade - Upgrade Pack to complete HBase
On Dec. 12, 2014, 12:08 a.m., Robert Levas wrote: ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java, lines 324-327 https://reviews.apache.org/r/28976/diff/1/?file=789939#file789939line324 I totally agree with the statement. Are you then just picking a random host from the cluster as the hostname? So do you think the double-check is too much? Can we just go with the non-host variant of the method for server-based tasks? That comment is in a block that happens when the first call returns a null/zero-length list. I'd prefer just the one call without host. Yes, right now the hack is to just pick any host. - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28976/#review64866 --- On Dec. 11, 2014, 10:07 p.m., Nate Cole wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28976/ --- (Updated Dec. 11, 2014, 10:07 p.m.) Review request for Ambari, Alejandro Fernandez, Robert Levas, and Tom Beerbower. Bugs: AMBARI-8675 https://issues.apache.org/jira/browse/AMBARI-8675 Repository: ambari Description --- PLEASE READ NOTES IF YOU'RE ASSIGNED TO THIS Adds HBase RU capability as well as: * Fix for Stage and UpgradeItem requests that bring back all tasks. * Fix for specifying a SERVICE_CHECK should look for the specific Role definition. This was causing numerous exceptions when trying to resolve the agent's response. * Added some code that works around the fact that a server side action must run on Ambari, but was never coming out of QUEUED. Rob: I know you're not part of rolling upgrade effort, but I had to make a fix to the ServerActionScheduler and db accessor - can you please review those parts? This is all part of the whole server action on ambari when it's not in the host table issue. It's easier for now to work around it then add ambari to the host table at install time (we need buy-in at a higher level for that. read: Mahadev). I added a comment that should get special attention in ServerActionScheduler.doWork(). Tom: for the Stage/Task stuff, it looks like Tasks/stage_id wasn't added as a key property, so a request for a Stage was bringing back all tests. My unit tests passed and accessing the endpoint looks good, but please check it out if you have a deployed cluster. The fix that Rob is looking at will be important for manual stages to work properly. (My no-op was failing to become PENDING, stuck as QUEUED). Alejandro: Please look at the agent and upgrade-pack additions. Thanks guys! I'd be happy to discuss any point in the review if need be. Diffs - ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessor.java 6ff365b ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java e8be3cc ambari-server/src/main/java/org/apache/ambari/server/api/services/UpgradeItemService.java a8a7640 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java b646c16 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 3bc5c4e ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java cf025b7 ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java 880c596 ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ColocatedGrouping.java f89645a ambari-server/src/main/resources/key_properties.json 0b4fa7b ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_master.py 6cd77e7 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_regionserver.py 8ce1e37 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/params.py a5b8549 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/upgrade.py PRE-CREATION ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 060470f ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java c55cde8 ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java f40c638 Diff: https://reviews.apache.org/r/28976/diff/ Testing --- Tests run: 2412, Failures: 0, Errors: 0, Skipped: 19 [INFO] [INFO] BUILD SUCCESS [INFO]
Re: Review Request 28976: Rolling Upgrade - Upgrade Pack to complete HBase
On Dec. 12, 2014, 12:08 a.m., Robert Levas wrote: ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java, lines 324-327 https://reviews.apache.org/r/28976/diff/1/?file=789939#file789939line324 I totally agree with the statement. Are you then just picking a random host from the cluster as the hostname? Nate Cole wrote: So do you think the double-check is too much? Can we just go with the non-host variant of the method for server-based tasks? That comment is in a block that happens when the first call returns a null/zero-length list. I'd prefer just the one call without host. Yes, right now the hack is to just pick any host. I think we can go with the _any host_ route. Feel free to make the change. - Robert --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28976/#review64866 --- On Dec. 11, 2014, 10:07 p.m., Nate Cole wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28976/ --- (Updated Dec. 11, 2014, 10:07 p.m.) Review request for Ambari, Alejandro Fernandez, Robert Levas, and Tom Beerbower. Bugs: AMBARI-8675 https://issues.apache.org/jira/browse/AMBARI-8675 Repository: ambari Description --- PLEASE READ NOTES IF YOU'RE ASSIGNED TO THIS Adds HBase RU capability as well as: * Fix for Stage and UpgradeItem requests that bring back all tasks. * Fix for specifying a SERVICE_CHECK should look for the specific Role definition. This was causing numerous exceptions when trying to resolve the agent's response. * Added some code that works around the fact that a server side action must run on Ambari, but was never coming out of QUEUED. Rob: I know you're not part of rolling upgrade effort, but I had to make a fix to the ServerActionScheduler and db accessor - can you please review those parts? This is all part of the whole server action on ambari when it's not in the host table issue. It's easier for now to work around it then add ambari to the host table at install time (we need buy-in at a higher level for that. read: Mahadev). I added a comment that should get special attention in ServerActionScheduler.doWork(). Tom: for the Stage/Task stuff, it looks like Tasks/stage_id wasn't added as a key property, so a request for a Stage was bringing back all tests. My unit tests passed and accessing the endpoint looks good, but please check it out if you have a deployed cluster. The fix that Rob is looking at will be important for manual stages to work properly. (My no-op was failing to become PENDING, stuck as QUEUED). Alejandro: Please look at the agent and upgrade-pack additions. Thanks guys! I'd be happy to discuss any point in the review if need be. Diffs - ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessor.java 6ff365b ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java e8be3cc ambari-server/src/main/java/org/apache/ambari/server/api/services/UpgradeItemService.java a8a7640 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java b646c16 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 3bc5c4e ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java cf025b7 ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java 880c596 ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ColocatedGrouping.java f89645a ambari-server/src/main/resources/key_properties.json 0b4fa7b ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_master.py 6cd77e7 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_regionserver.py 8ce1e37 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/params.py a5b8549 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/upgrade.py PRE-CREATION ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 060470f ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java c55cde8 ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java f40c638 Diff: https://reviews.apache.org/r/28976/diff/ Testing --- Tests run: 2412, Failures: 0, Errors: 0, Skipped: 19 [INFO]
Re: Review Request 28976: Rolling Upgrade - Upgrade Pack to complete HBase
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28976/#review64904 --- ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/upgrade.py https://reviews.apache.org/r/28976/#comment107728 I will borrow this style of having an upgrade.py file for other services. ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml https://reviews.apache.org/r/28976/#comment107729 Does HBASE_REGIONSERVER also have to be added to the order element, just like HBASE_MASTER was? - Alejandro Fernandez On Dec. 12, 2014, 3:07 a.m., Nate Cole wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28976/ --- (Updated Dec. 12, 2014, 3:07 a.m.) Review request for Ambari, Alejandro Fernandez, Robert Levas, and Tom Beerbower. Bugs: AMBARI-8675 https://issues.apache.org/jira/browse/AMBARI-8675 Repository: ambari Description --- PLEASE READ NOTES IF YOU'RE ASSIGNED TO THIS Adds HBase RU capability as well as: * Fix for Stage and UpgradeItem requests that bring back all tasks. * Fix for specifying a SERVICE_CHECK should look for the specific Role definition. This was causing numerous exceptions when trying to resolve the agent's response. * Added some code that works around the fact that a server side action must run on Ambari, but was never coming out of QUEUED. Rob: I know you're not part of rolling upgrade effort, but I had to make a fix to the ServerActionScheduler and db accessor - can you please review those parts? This is all part of the whole server action on ambari when it's not in the host table issue. It's easier for now to work around it then add ambari to the host table at install time (we need buy-in at a higher level for that. read: Mahadev). I added a comment that should get special attention in ServerActionScheduler.doWork(). Tom: for the Stage/Task stuff, it looks like Tasks/stage_id wasn't added as a key property, so a request for a Stage was bringing back all tests. My unit tests passed and accessing the endpoint looks good, but please check it out if you have a deployed cluster. The fix that Rob is looking at will be important for manual stages to work properly. (My no-op was failing to become PENDING, stuck as QUEUED). Alejandro: Please look at the agent and upgrade-pack additions. Thanks guys! I'd be happy to discuss any point in the review if need be. Diffs - ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessor.java 6ff365b ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java e8be3cc ambari-server/src/main/java/org/apache/ambari/server/api/services/UpgradeItemService.java a8a7640 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java b646c16 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 3bc5c4e ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java cf025b7 ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java 880c596 ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ColocatedGrouping.java f89645a ambari-server/src/main/resources/key_properties.json 0b4fa7b ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_master.py 6cd77e7 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_regionserver.py 8ce1e37 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/params.py a5b8549 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/upgrade.py PRE-CREATION ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 060470f ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java c55cde8 ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java f40c638 Diff: https://reviews.apache.org/r/28976/diff/ Testing --- Tests run: 2412, Failures: 0, Errors: 0, Skipped: 19 [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 22:15.071s [INFO] Finished at: Thu Dec 11 21:52:42 EST 2014 [INFO] Final Memory: 31M/327M [INFO]
Re: Review Request 28976: Rolling Upgrade - Upgrade Pack to complete HBase
On Dec. 12, 2014, 12:08 a.m., Robert Levas wrote: ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java, lines 324-327 https://reviews.apache.org/r/28976/diff/1/?file=789939#file789939line324 I totally agree with the statement. Are you then just picking a random host from the cluster as the hostname? Nate Cole wrote: So do you think the double-check is too much? Can we just go with the non-host variant of the method for server-based tasks? That comment is in a block that happens when the first call returns a null/zero-length list. I'd prefer just the one call without host. Yes, right now the hack is to just pick any host. Robert Levas wrote: I think we can go with the _any host_ route. Feel free to make the change. Thank you so much for reviewing this - I'll get things working for now and open a new JIRA to address for real. - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28976/#review64866 --- On Dec. 11, 2014, 10:07 p.m., Nate Cole wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28976/ --- (Updated Dec. 11, 2014, 10:07 p.m.) Review request for Ambari, Alejandro Fernandez, Robert Levas, and Tom Beerbower. Bugs: AMBARI-8675 https://issues.apache.org/jira/browse/AMBARI-8675 Repository: ambari Description --- PLEASE READ NOTES IF YOU'RE ASSIGNED TO THIS Adds HBase RU capability as well as: * Fix for Stage and UpgradeItem requests that bring back all tasks. * Fix for specifying a SERVICE_CHECK should look for the specific Role definition. This was causing numerous exceptions when trying to resolve the agent's response. * Added some code that works around the fact that a server side action must run on Ambari, but was never coming out of QUEUED. Rob: I know you're not part of rolling upgrade effort, but I had to make a fix to the ServerActionScheduler and db accessor - can you please review those parts? This is all part of the whole server action on ambari when it's not in the host table issue. It's easier for now to work around it then add ambari to the host table at install time (we need buy-in at a higher level for that. read: Mahadev). I added a comment that should get special attention in ServerActionScheduler.doWork(). Tom: for the Stage/Task stuff, it looks like Tasks/stage_id wasn't added as a key property, so a request for a Stage was bringing back all tests. My unit tests passed and accessing the endpoint looks good, but please check it out if you have a deployed cluster. The fix that Rob is looking at will be important for manual stages to work properly. (My no-op was failing to become PENDING, stuck as QUEUED). Alejandro: Please look at the agent and upgrade-pack additions. Thanks guys! I'd be happy to discuss any point in the review if need be. Diffs - ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessor.java 6ff365b ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java e8be3cc ambari-server/src/main/java/org/apache/ambari/server/api/services/UpgradeItemService.java a8a7640 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java b646c16 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 3bc5c4e ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java cf025b7 ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java 880c596 ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ColocatedGrouping.java f89645a ambari-server/src/main/resources/key_properties.json 0b4fa7b ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_master.py 6cd77e7 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_regionserver.py 8ce1e37 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/params.py a5b8549 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/upgrade.py PRE-CREATION ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 060470f ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java c55cde8 ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java f40c638 Diff:
Re: Review Request 28976: Rolling Upgrade - Upgrade Pack to complete HBase
On Dec. 12, 2014, 9:37 a.m., Alejandro Fernandez wrote: ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml, line 298 https://reviews.apache.org/r/28976/diff/1/?file=789946#file789946line298 Does HBASE_REGIONSERVER also have to be added to the order element, just like HBASE_MASTER was? There was one, but actually just saw it was named incorrectly. Fixing now. Thanks! - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28976/#review64904 --- On Dec. 11, 2014, 10:07 p.m., Nate Cole wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28976/ --- (Updated Dec. 11, 2014, 10:07 p.m.) Review request for Ambari, Alejandro Fernandez, Robert Levas, and Tom Beerbower. Bugs: AMBARI-8675 https://issues.apache.org/jira/browse/AMBARI-8675 Repository: ambari Description --- PLEASE READ NOTES IF YOU'RE ASSIGNED TO THIS Adds HBase RU capability as well as: * Fix for Stage and UpgradeItem requests that bring back all tasks. * Fix for specifying a SERVICE_CHECK should look for the specific Role definition. This was causing numerous exceptions when trying to resolve the agent's response. * Added some code that works around the fact that a server side action must run on Ambari, but was never coming out of QUEUED. Rob: I know you're not part of rolling upgrade effort, but I had to make a fix to the ServerActionScheduler and db accessor - can you please review those parts? This is all part of the whole server action on ambari when it's not in the host table issue. It's easier for now to work around it then add ambari to the host table at install time (we need buy-in at a higher level for that. read: Mahadev). I added a comment that should get special attention in ServerActionScheduler.doWork(). Tom: for the Stage/Task stuff, it looks like Tasks/stage_id wasn't added as a key property, so a request for a Stage was bringing back all tests. My unit tests passed and accessing the endpoint looks good, but please check it out if you have a deployed cluster. The fix that Rob is looking at will be important for manual stages to work properly. (My no-op was failing to become PENDING, stuck as QUEUED). Alejandro: Please look at the agent and upgrade-pack additions. Thanks guys! I'd be happy to discuss any point in the review if need be. Diffs - ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessor.java 6ff365b ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java e8be3cc ambari-server/src/main/java/org/apache/ambari/server/api/services/UpgradeItemService.java a8a7640 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java b646c16 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 3bc5c4e ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java cf025b7 ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java 880c596 ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ColocatedGrouping.java f89645a ambari-server/src/main/resources/key_properties.json 0b4fa7b ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_master.py 6cd77e7 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_regionserver.py 8ce1e37 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/params.py a5b8549 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/upgrade.py PRE-CREATION ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 060470f ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java c55cde8 ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java f40c638 Diff: https://reviews.apache.org/r/28976/diff/ Testing --- Tests run: 2412, Failures: 0, Errors: 0, Skipped: 19 [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 22:15.071s [INFO] Finished at: Thu Dec 11 21:52:42 EST 2014 [INFO] Final Memory: 31M/327M [INFO] Thanks, Nate Cole
Re: Review Request 28976: Rolling Upgrade - Upgrade Pack to complete HBase
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28976/ --- (Updated Dec. 12, 2014, 9:53 a.m.) Review request for Ambari, Alejandro Fernandez, Robert Levas, and Tom Beerbower. Changes --- Updated diff for comments Bugs: AMBARI-8675 https://issues.apache.org/jira/browse/AMBARI-8675 Repository: ambari Description --- PLEASE READ NOTES IF YOU'RE ASSIGNED TO THIS Adds HBase RU capability as well as: * Fix for Stage and UpgradeItem requests that bring back all tasks. * Fix for specifying a SERVICE_CHECK should look for the specific Role definition. This was causing numerous exceptions when trying to resolve the agent's response. * Added some code that works around the fact that a server side action must run on Ambari, but was never coming out of QUEUED. Rob: I know you're not part of rolling upgrade effort, but I had to make a fix to the ServerActionScheduler and db accessor - can you please review those parts? This is all part of the whole server action on ambari when it's not in the host table issue. It's easier for now to work around it then add ambari to the host table at install time (we need buy-in at a higher level for that. read: Mahadev). I added a comment that should get special attention in ServerActionScheduler.doWork(). Tom: for the Stage/Task stuff, it looks like Tasks/stage_id wasn't added as a key property, so a request for a Stage was bringing back all tests. My unit tests passed and accessing the endpoint looks good, but please check it out if you have a deployed cluster. The fix that Rob is looking at will be important for manual stages to work properly. (My no-op was failing to become PENDING, stuck as QUEUED). Alejandro: Please look at the agent and upgrade-pack additions. Thanks guys! I'd be happy to discuss any point in the review if need be. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessor.java 6ff365b ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java e8be3cc ambari-server/src/main/java/org/apache/ambari/server/api/services/UpgradeItemService.java a8a7640 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java b646c16 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 3bc5c4e ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java cf025b7 ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java 880c596 ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ColocatedGrouping.java f89645a ambari-server/src/main/resources/key_properties.json 0b4fa7b ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_master.py 6cd77e7 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_regionserver.py 8ce1e37 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/params.py a5b8549 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/upgrade.py PRE-CREATION ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 060470f ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java c55cde8 ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java f40c638 Diff: https://reviews.apache.org/r/28976/diff/ Testing --- Tests run: 2412, Failures: 0, Errors: 0, Skipped: 19 [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 22:15.071s [INFO] Finished at: Thu Dec 11 21:52:42 EST 2014 [INFO] Final Memory: 31M/327M [INFO] Thanks, Nate Cole
Re: Review Request 28976: Rolling Upgrade - Upgrade Pack to complete HBase
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28976/#review64907 --- Ship it! Nicely done - Alejandro Fernandez On Dec. 12, 2014, 2:53 p.m., Nate Cole wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28976/ --- (Updated Dec. 12, 2014, 2:53 p.m.) Review request for Ambari, Alejandro Fernandez, Robert Levas, and Tom Beerbower. Bugs: AMBARI-8675 https://issues.apache.org/jira/browse/AMBARI-8675 Repository: ambari Description --- PLEASE READ NOTES IF YOU'RE ASSIGNED TO THIS Adds HBase RU capability as well as: * Fix for Stage and UpgradeItem requests that bring back all tasks. * Fix for specifying a SERVICE_CHECK should look for the specific Role definition. This was causing numerous exceptions when trying to resolve the agent's response. * Added some code that works around the fact that a server side action must run on Ambari, but was never coming out of QUEUED. Rob: I know you're not part of rolling upgrade effort, but I had to make a fix to the ServerActionScheduler and db accessor - can you please review those parts? This is all part of the whole server action on ambari when it's not in the host table issue. It's easier for now to work around it then add ambari to the host table at install time (we need buy-in at a higher level for that. read: Mahadev). I added a comment that should get special attention in ServerActionScheduler.doWork(). Tom: for the Stage/Task stuff, it looks like Tasks/stage_id wasn't added as a key property, so a request for a Stage was bringing back all tests. My unit tests passed and accessing the endpoint looks good, but please check it out if you have a deployed cluster. The fix that Rob is looking at will be important for manual stages to work properly. (My no-op was failing to become PENDING, stuck as QUEUED). Alejandro: Please look at the agent and upgrade-pack additions. Thanks guys! I'd be happy to discuss any point in the review if need be. Diffs - ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessor.java 6ff365b ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java e8be3cc ambari-server/src/main/java/org/apache/ambari/server/api/services/UpgradeItemService.java a8a7640 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java b646c16 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 3bc5c4e ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java cf025b7 ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java 880c596 ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ColocatedGrouping.java f89645a ambari-server/src/main/resources/key_properties.json 0b4fa7b ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_master.py 6cd77e7 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_regionserver.py 8ce1e37 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/params.py a5b8549 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/upgrade.py PRE-CREATION ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 060470f ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java c55cde8 ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java f40c638 Diff: https://reviews.apache.org/r/28976/diff/ Testing --- Tests run: 2412, Failures: 0, Errors: 0, Skipped: 19 [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 22:15.071s [INFO] Finished at: Thu Dec 11 21:52:42 EST 2014 [INFO] Final Memory: 31M/327M [INFO] Thanks, Nate Cole
Re: Review Request 28976: Rolling Upgrade - Upgrade Pack to complete HBase
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28976/#review64908 --- Ship it! ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ColocatedGrouping.java https://reviews.apache.org/r/28976/#comment107732 Still need the TODO here? ambari-server/src/main/resources/key_properties.json https://reviews.apache.org/r/28976/#comment107733 I guess this was missed from before. I'm surprised this didn't cause problems... or maybe it did. How did you discover this was missing? - Tom Beerbower On Dec. 12, 2014, 2:53 p.m., Nate Cole wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28976/ --- (Updated Dec. 12, 2014, 2:53 p.m.) Review request for Ambari, Alejandro Fernandez, Robert Levas, and Tom Beerbower. Bugs: AMBARI-8675 https://issues.apache.org/jira/browse/AMBARI-8675 Repository: ambari Description --- PLEASE READ NOTES IF YOU'RE ASSIGNED TO THIS Adds HBase RU capability as well as: * Fix for Stage and UpgradeItem requests that bring back all tasks. * Fix for specifying a SERVICE_CHECK should look for the specific Role definition. This was causing numerous exceptions when trying to resolve the agent's response. * Added some code that works around the fact that a server side action must run on Ambari, but was never coming out of QUEUED. Rob: I know you're not part of rolling upgrade effort, but I had to make a fix to the ServerActionScheduler and db accessor - can you please review those parts? This is all part of the whole server action on ambari when it's not in the host table issue. It's easier for now to work around it then add ambari to the host table at install time (we need buy-in at a higher level for that. read: Mahadev). I added a comment that should get special attention in ServerActionScheduler.doWork(). Tom: for the Stage/Task stuff, it looks like Tasks/stage_id wasn't added as a key property, so a request for a Stage was bringing back all tests. My unit tests passed and accessing the endpoint looks good, but please check it out if you have a deployed cluster. The fix that Rob is looking at will be important for manual stages to work properly. (My no-op was failing to become PENDING, stuck as QUEUED). Alejandro: Please look at the agent and upgrade-pack additions. Thanks guys! I'd be happy to discuss any point in the review if need be. Diffs - ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessor.java 6ff365b ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java e8be3cc ambari-server/src/main/java/org/apache/ambari/server/api/services/UpgradeItemService.java a8a7640 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java b646c16 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 3bc5c4e ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java cf025b7 ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java 880c596 ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ColocatedGrouping.java f89645a ambari-server/src/main/resources/key_properties.json 0b4fa7b ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_master.py 6cd77e7 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_regionserver.py 8ce1e37 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/params.py a5b8549 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/upgrade.py PRE-CREATION ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 060470f ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java c55cde8 ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java f40c638 Diff: https://reviews.apache.org/r/28976/diff/ Testing --- Tests run: 2412, Failures: 0, Errors: 0, Skipped: 19 [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 22:15.071s [INFO] Finished at: Thu Dec 11 21:52:42 EST 2014 [INFO] Final Memory: 31M/327M [INFO] Thanks,
Re: Review Request 28976: Rolling Upgrade - Upgrade Pack to complete HBase
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28976/#review64866 --- Ship it! I'd be happy to work on a proper solution to the hostname issue. Ideally the host does not matter as long as the task is tagged as a server-side action. ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java https://reviews.apache.org/r/28976/#comment107660 Wouldn't it be safer to do the following? ``` if ((tasks == null) || tasks.isEmpty()) ``` Maybe `getTasksByHostRoldAndStatus` never returns `null` and this is overkill, but it make me feel better whenever I see protection against `null` :) ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java https://reviews.apache.org/r/28976/#comment107661 I totally agree with the statement. Are you then just picking a random host from the cluster as the hostname? - Robert Levas On Dec. 11, 2014, 10:07 p.m., Nate Cole wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28976/ --- (Updated Dec. 11, 2014, 10:07 p.m.) Review request for Ambari, Alejandro Fernandez, Robert Levas, and Tom Beerbower. Bugs: AMBARI-8675 https://issues.apache.org/jira/browse/AMBARI-8675 Repository: ambari Description --- PLEASE READ NOTES IF YOU'RE ASSIGNED TO THIS Adds HBase RU capability as well as: * Fix for Stage and UpgradeItem requests that bring back all tasks. * Fix for specifying a SERVICE_CHECK should look for the specific Role definition. This was causing numerous exceptions when trying to resolve the agent's response. * Added some code that works around the fact that a server side action must run on Ambari, but was never coming out of QUEUED. Rob: I know you're not part of rolling upgrade effort, but I had to make a fix to the ServerActionScheduler and db accessor - can you please review those parts? This is all part of the whole server action on ambari when it's not in the host table issue. It's easier for now to work around it then add ambari to the host table at install time (we need buy-in at a higher level for that. read: Mahadev). I added a comment that should get special attention in ServerActionScheduler.doWork(). Tom: for the Stage/Task stuff, it looks like Tasks/stage_id wasn't added as a key property, so a request for a Stage was bringing back all tests. My unit tests passed and accessing the endpoint looks good, but please check it out if you have a deployed cluster. The fix that Rob is looking at will be important for manual stages to work properly. (My no-op was failing to become PENDING, stuck as QUEUED). Alejandro: Please look at the agent and upgrade-pack additions. Thanks guys! I'd be happy to discuss any point in the review if need be. Diffs - ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessor.java 6ff365b ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java e8be3cc ambari-server/src/main/java/org/apache/ambari/server/api/services/UpgradeItemService.java a8a7640 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java b646c16 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 3bc5c4e ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java cf025b7 ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java 880c596 ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ColocatedGrouping.java f89645a ambari-server/src/main/resources/key_properties.json 0b4fa7b ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_master.py 6cd77e7 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_regionserver.py 8ce1e37 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/params.py a5b8549 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/package/scripts/upgrade.py PRE-CREATION ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 060470f ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java c55cde8 ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java f40c638 Diff: https://reviews.apache.org/r/28976/diff/ Testing --- Tests run: 2412, Failures: 0, Errors:
