> On Jan. 15, 2016, 3:45 p.m., Jonathan Hurley wrote: > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java, > > lines 106-109 > > <https://reviews.apache.org/r/42337/diff/1/?file=1197891#file1197891line106> > > > > I'd add some documentation around this change to say why it's being > > done this way. It'll make it clearer to anyone looking at it in the future > > who wonders why not just create a single stage for the tasks. > > Alejandro Fernandez wrote: > Certainly. This can be optimized to minimize the number of stages but I > decided not to for the sake of simplicity and the fact that most pre-upgrade > and post-upgrade elements have only one stage. > E.g., if 2 tasks have to run in pre-upgrade and they each run on the same > set of hosts, then only one stage is needed for both tasks. However, if they > run on distinct sets of hosts, then they must be different stages. > I didn't want to have to calculate if the set of hosts were not > equivalent, and then break it up into optimal groupings of hosts.
Spoke to Jonathan. Having each configure task (which occur in adjacent elements of 6 or more) in its own stage is increasing the DB footprint. Instead of adding more voodoo to Grouping.java, I'll explicitly annotate in the XML file how the tasks should be broken into stages by introducing an attribute to a task, e.g., stage="synchronized|group". By default, tasks are grouped into the same stage, but one task can explictly call out to be done in its own stage. Every synchronized task delineates the stages. - Alejandro ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/42337/#review114711 ----------------------------------------------------------- On Jan. 15, 2016, 2:45 a.m., Alejandro Fernandez wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/42337/ > ----------------------------------------------------------- > > (Updated Jan. 15, 2016, 2:45 a.m.) > > > Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan > Hurley, Jayush Luniya, and Nate Cole. > > > Bugs: AMBARI-14678 > https://issues.apache.org/jira/browse/AMBARI-14678 > > > Repository: ambari > > > Description > ------- > > STR: > HDP 2.2 > Multiple Oozie Servers > RU to HDP 2.3 > > The pre-upgrade directive for Oozie has > ``` > <pre-upgrade> > <task xsi:type="execute" hosts="all" summary="Shut down all Oozie servers"> > <script>scripts/oozie_server.py</script> > <function>stop</function> > </task> > > <task xsi:type="execute" hosts="any" summary="Upgrading the database and > creating a new sharelib"> > <script>scripts/oozie_server_upgrade.py</script> > <function>upgrade_oozie_database_and_sharelib</function> > </task> > </pre-upgrade> > ``` > > Notice that the first task runs on all hosts, and the second task runs on > exactly one random host. The problem is that the scheduler runs both tasks in > one stage on the union of these hosts. Therefore, the 2nd task runs on the > wrong set of hosts. > > > Diffs > ----- > > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java > aa852bd > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/StageWrapperBuilder.java > b0df83f > > ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java > 7e61942 > > ambari-server/src/test/resources/stacks/HDP/2.0.7/services/OOZIE/metainfo.xml > PRE-CREATION > ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test.xml > 4f71b4a > > Diff: https://reviews.apache.org/r/42337/diff/ > > > Testing > ------- > > Verified during RU from HDP 2.2 to 2.3 with multiple Oozie Servers. > > Waiting for unit test results. > > > Thanks, > > Alejandro Fernandez > >
