> On March 24, 2017, 10:08 a.m., Jonathan Hurley wrote: > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ParallelScheduler.java > > Lines 56-57 (patched) > > <https://reviews.apache.org/r/57792/diff/8/?file=1673723#file1673723line56> > > > > I thought I had flagged this before and you agreed it was a problem - > > doesn't this mean get a list of sets with each set containing only 1 item? > > The returned collection should essentially be a singleton list which > > contains a single set of all hosts.
Let me be clearer here: - If the PS is omitted, then this code is correct - it should not be parallelized, so you should force it to 1 - If the PS is specified and the MaxDegreeOP is set to 0, then this should do what I suggested, which is run them all in parallel (singleton list of 1 set with all items in set) - Jonathan ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57792/#review170011 ----------------------------------------------------------- On March 23, 2017, 9:29 p.m., Alejandro Fernandez wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/57792/ > ----------------------------------------------------------- > > (Updated March 23, 2017, 9:29 p.m.) > > > Review request for Ambari, Jonathan Hurley, Nate Cole, Sumit Mohanty, and Sid > Wagle. > > > Bugs: AMBARI-20490 > https://issues.apache.org/jira/browse/AMBARI-20490 > > > Repository: ambari > > > Description > ------- > > On large-scale clusters over 1000 nodes, performance during Express Upgrade > can be very slow and it takes more time to fix potential errors. > We can remedy these with the following fixes: > > * "hdp-select set all" is currently ran in the ClusterGroup with a single > stage instead of batching. This can cause Ambari Server to be the bottleneck > when processing 1000+ requests. Allow the execute-stage inside a ClusterGroup > to use batching. > > * If some hosts have symlinks or dirs that don't belong in /usr/hdp/ then it > will prevent the hosts from running "hdp-select versions". We can find these > problematic hosts with an alert. > > * There is currently no upgrade path from HDP 2.2 directly to 2.5. Doing a > 2-step upgrade is very time consuming (only for branch 2.4 since HDP 2.2 no > longer supported in Ambari 2.5). > > > Diffs > ----- > > > ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py > 20b4cbd > > ambari-server/src/main/java/org/apache/ambari/server/checks/AtlasPresenceCheck.java > 8556436 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java > ca9ce07 > > ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java > ee5451f > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java > c45ec24 > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java > 8003223 > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ParallelScheduler.java > 6004a12 > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/StageWrapper.java > 669d50f > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/StageWrapperBuilder.java > 3cf3f8b > ambari-server/src/main/resources/alerts.json 2559b3a > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-properties.xml > ca14c1f > ambari-server/src/main/resources/host_scripts/alert_version_select.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml > 28ca939 > > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.4.xml > 9aff1a9 > > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml > adccac0 > > ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.4.xml > 3c0104b > > ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml > cd63651 > > ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml > 6f7fbd0 > > ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.6.xml > 063b362 > > ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/nonrolling-upgrade-2.6.xml > 4a641ec > ambari-server/src/main/resources/upgrade-pack.xsd 1f11aa1 > > ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java > f1af66f > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java > bf6c41a > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java > 3e70989 > > ambari-server/src/test/java/org/apache/ambari/server/metadata/AgentAlertDefinitionsTest.java > cbc5e69 > > ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java > 8e5ad0a > > ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test_batch.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_to_new_stack.xml > fe387f3 > > > Diff: https://reviews.apache.org/r/57792/diff/8/ > > > Testing > ------- > > mvn clean test > > > Thanks, > > Alejandro Fernandez > >
