[
https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16196544#comment-16196544
]
Eric Yang commented on YARN-7202:
---------------------------------
[~jianhe] ServiceClient code is partially completed in YARN-7202.
"updateService" code is not quite testable because service operation and config
change sequence are inverse of what they are supposed to be.
Prior to this patch, the PUT method is:
If Service object should be in STARTED state, then return.
If Service object should be in STOPPED state, then return.
If Service object should increase or decrease containers, perform the operation.
The problem arises, if someone retrieve Service status object via GET
/ws/v1/services/{service_name}, and change number of container for the service.
It will not perform the container changes. The code got to this point because
there is no separation between configuration, and status. If status object is
reused to perform configuration changes, then the code doesn't perform what it
was designed to do. For clarity, flex number of container operation should be
considered as configuration changes because even existing containers may need
to have information of newly spawning containers. Hence, it is best that we
treat flexing container numbers as configuration change, and restart all
containers with in the service.
Giving the above reasons, the proper design should be:
If Service config changed, save configuration.
If Service should increase/decrease containers, perform flex operation.
If Service should be in STARTED state, then start containers.
If Service should be in STOPPED state, then stop containers.
At this point, ServiceClient is not written to support the proper sequence of
operation to perform. I only made changes to ApiServer updateService method to
contain the scope of required changes. Therefore, I use mock object here to
allow TestApiService to pass without making a failure test case to block other
development. ServiceClient will not pass the test code at this point, but
ApiServer code can pass the test with improved logic. This is the reason that
mock object is used here. In YARN-7215, after spec and status are separated
two calls. I added additional logic in ServiceClient code to have ability to
make service config changes. In YARN-7215, you can see that TestApiService is
switched to the real ServiceClient, and both positive and negative test cases
passed.
The dependency are required to import MiniCluster. For some reason, I think
the code refactoring has some issues. Hadoop-yarn-service-api is not a sibling
project to hadoop-yarn-services, or hadoop-yarn-services-core. Therefore, it
doesn't inherit MiniCluster imports. This is the reason that test jar files
are imported again.
I think YARN-7202, YARN-7216, and YARN-7215 should be one transaction. Giving
degree of complexity in the design bug, it is good that we discuss the reasons
for these changes. The issues are raised out of order due to late discovery of
the bugs in the inherited code. Hence, the course correction is only made in
later JIRAs.
Hope this clarifies that the end goal of this JIRA is met, but the proper code
will only happen after YARN-7216 and YARN-7215 are committed.
> End-to-end UT for api-server
> ----------------------------
>
> Key: YARN-7202
> URL: https://issues.apache.org/jira/browse/YARN-7202
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Jian He
> Assignee: Eric Yang
> Attachments: YARN-7202.yarn-native-services.001.patch,
> YARN-7202.yarn-native-services.002.patch,
> YARN-7202.yarn-native-services.003.patch,
> YARN-7202.yarn-native-services.004.patch,
> YARN-7202.yarn-native-services.005.patch,
> YARN-7202.yarn-native-services.006.patch,
> YARN-7202.yarn-native-services.007.patch,
> YARN-7202.yarn-native-services.008.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
