> On Jan. 8, 2016, 1:49 p.m., Jonathan Hurley wrote:
> > Why go through all of these code changes to accomplish this? It seems like
> > you could just have removed the `parallel-scheduler` element from the
> > upgrade XML:
> >
> > ```
> > <group xsi:type="restart" name="HIVE" title="Hive">
> > <service-check>false</service-check>
> > <skippable>true</skippable>
> > <supports-auto-skip-failure>false</supports-auto-skip-failure>
> > <parallel-scheduler/>
> > <service name="HIVE">
> > <component>HIVE_METASTORE</component>
> > <component>HIVE_SERVER</component>
> > <component>WEBHCAT_SERVER</component>
> > <component>HIVE_CLIENT</component>
> > <component>HCAT</component>
> > </service>
> > </group>
> > ```
>
> Nate Cole wrote:
> +1 for this
>
> Alejandro Fernandez wrote:
> I want the group to still be in parallel, but only Hive Server and
> WebHCat server to be serial. Breaking it up into 2 separate groups seemed too
> clunky. If the default behavior of parallel-scheduler changes, then I still
> ensure that those 2 components run sequentially.
>
> Jonathan Hurley wrote:
> That's what the groupings are for though. I'd expect this kind of change
> to be an XML-only change since it's so simple. Seems like we're introducing a
> lot of new code which overcomplicates the solution.
>
> Alejandro Fernandez wrote:
> I thought about that, but didn't want the EU/RU wizard to then show
> multiple groups for Hive.
>
> Jonathan Hurley wrote:
> I think this is a matter of trading off between a cleaner XML with a
> single group vs having additional complex code for calculating the maximum
> degree of parallelism. My feeling is that even though it's a neat feature,
> it's not something required to fix this problem. Fixing it in the XML seems
> to be more intuitive since it will be represented to the user as 2 different
> groups.
>
> If others feel strongly that we should keep Hive as a single group, then
> I'm OK with it. I just didn't think this much code was necessary to fix the
> problem.
>
> Nate Cole wrote:
> I would prefer keeping the code simpler. The whole point of the upgrade
> pack "verbosity" is to make it clear to someone viewing it what is going to
> happen. That there's two hive groups being shown isn't a big deal. "Hive"
> and "Hive Clients."
While it would have been better to have an ability to override, I agree that it
is less readable based on how the upgrade pack needs to be authored. It would
have been better if groups could have sub-groups and we can set overrides in
sub-groups. That way we wouldn't see separate groups but internally override
group-level properties. For now maybe we can just keep it simple and leave it
as separate groups.
<group name="Hive">
<parallel-scheduler>
<max-degree-of-parallelism>100</max-degree-of-parallelism>
</parallel-scheduler>
<sub-group name="Hive">
<parallel-scheduler>
<max-degree-of-parallelism>1</max-degree-of-parallelism>
</parallel-scheduler>
<sub-group>
<sub-group name="Hive Client">
</sub-group>
</group>
- Jayush
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42038/#review113472
-----------------------------------------------------------
On Jan. 7, 2016, 10:18 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42038/
> -----------------------------------------------------------
>
> (Updated Jan. 7, 2016, 10:18 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan
> Hurley, Jayush Luniya, and Nate Cole.
>
>
> Bugs: AMBARI-14581
> https://issues.apache.org/jira/browse/AMBARI-14581
>
>
> Repository: ambari
>
>
> Description
> -------
>
> During an Express Upgrade, all of the tasks for Hive Server and Hive
> MetaStore run in parallel, so if these components have multiple hosts, then
> running in parallel can cause errors during the upgrade.
> E.g., Hive Server copies tarballs, which can only run one at a time. Hive
> MetaStore updates the schema, and only one can run at a time.
>
>
> Diffs
> -----
>
>
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
> aa852bd
>
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/StageWrapperBuilder.java
> b0df83f
>
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Task.java
> 60091a6
>
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml
> a46a5cd
>
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml
> e57b13f
>
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml
> b1c3739
>
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml
> ba2578b
>
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
> 7e61942
>
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/UpgradePackTest.java
> 15be8b4
>
> ambari-server/src/test/resources/stacks/HDP/2.0.7/services/HIVE/metainfo.xml
> 35a4d05
>
> ambari-server/src/test/resources/stacks/HDP/2.0.7/services/YARN/metainfo.xml
> beee6a0
>
> ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test_nonrolling.xml
> 9f57ca8
>
> Diff: https://reviews.apache.org/r/42038/diff/
>
>
> Testing
> -------
>
> Ran unit tests in UpgradeHelperTest.java and UpgradePackTest.java
> Verified with EU from HDP 2.2->2.3 with multiple Hive MetaStores.
>
> Waiting for unit test results.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
