[
https://issues.apache.org/jira/browse/YARN-7512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16401140#comment-16401140
]
Wangda Tan commented on YARN-7512:
----------------------------------
Thanks [~csingh] for working on this design, a couple of questions/comments.
1) Regarding to upgrade, from my understanding, it will be more valueable to
upgrade only for component's artifact and component's replicas.
For other part, since we have environment subsistution, it is possible that a
change of global environment could cause all dependencies upgraded. For example:
{code:java}
Service:
name: my-service.
environments:
env1: value1
env2: value2
components:
component:
name: comp1
launch_command: "python $env1 ..."
artifact: $env2-x-y-z
component:
name: comp2
launch_command: "app $env2 ..."
{code}
Change of $env1/$env2 could cause all component needs to be upgraded, I'm not
sure if that is desired state.
Talked to [~csingh], existing upgrade only supports artifact, and all other
fields will be silently ignored. Probably we should rename the "upgrade" to
something like "set-image"?
2) Regarding to existing interactions between user and native services to do
updates. IIUC, this is workflow proposed in the design doc, plz correct me if I
was wrong:
a. User upload a new upgrade spec to HDFS.
b. User call upgrade service to read the new spec.
c. User call upgrade component(s) to update components and instances.
d. User manually set finalize or YARN will automatically set finalize of
services.
First of all, I'm not sure if changing service state to UPGRADING could bring
and benefits. IMHO, it is confusing to let user manual finalize an upgrade but
not all components are upgraded.
And a/b look like a burden to end user. If user needs to call c anyway, why we
need a/b.
My proposal is to keep APIs to only upgrade *artifact* of component and
instance of components:
{code:java}
Response upgradeComponentArtifact(ServiceName, ComponentName, NewArtifactName,
[LowerIndexBound, HigherIndexBound])
{code}
The {{[LowerIndexBound, HigherIndexBound]}} is useful when user want to verify
the artifact works instead of bring down all instances. By default we will
upgrade all instances of the component.
Once API server received this, this operation will be persisted, and component
transferred to UPGRADING. And once all requested instances upgraded,
component's state will be updated to READY again.
3) Several future requirement to consider:
a. What's the rollback story? It might be good to list history artifacts of
instances and timestamp. Existing design doesn't support rollback to a specific
version.
b. What's the AM restart story? From Chandni now AM doesn't persistent upgrade
status of containers. We may need to consider this.
Thoughts?
cc:[~gsaha]/[~billie.rinaldi].
> Support service upgrade via YARN Service API and CLI
> ----------------------------------------------------
>
> Key: YARN-7512
> URL: https://issues.apache.org/jira/browse/YARN-7512
> Project: Hadoop YARN
> Issue Type: New Feature
> Reporter: Gour Saha
> Assignee: Chandni Singh
> Priority: Major
> Fix For: yarn-native-services
>
> Attachments: _In-Place Upgrade of Long-Running Applications in
> YARN_v1.pdf, _In-Place Upgrade of Long-Running Applications in YARN_v2.pdf
>
>
> YARN Service API and CLI needs to support service (and containers) upgrade in
> line with what Slider supported in SLIDER-787
> (http://slider.incubator.apache.org/docs/slider_specs/application_pkg_upgrade.html)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
