[
https://issues.apache.org/jira/browse/FLINK-16194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zili Chen closed FLINK-16194.
-----------------------------
Resolution: Fixed
master via (reverse order)
dd8ce6c19eefeaf60c085365fe8bea960c1dace8
f6ad9685bcb8365277ffc2fa3fe1d54b2d445a28
10f53290dd07a64013623d7269c9d599dd7c2482
d29628ca6d64581f8627f81e81f05fa457a6b68d
4d8281a3c89514e16a923e20440095e974b9d91d
7f19c9cf14162c0f1b49e0b7a9bd7a0134cc7e3e
a50435d443395141fc526c6584ec9841e5968713
22735e3f3fdafc86b9f3c20b361a3830b30d44e3
8c7a17e6ef7e3da884e377437563d2508193a56d
20d0990f54272039e033f308cbb8a81fd0c7c000
0743b437c764a20b72ba4b14ad1e8f08755c2108
11fa0283d2abf7bfeca1a846a8163174dbfc3080
> Refactor the Kubernetes decorator design
> ----------------------------------------
>
> Key: FLINK-16194
> URL: https://issues.apache.org/jira/browse/FLINK-16194
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / Kubernetes
> Affects Versions: 1.10.0
> Reporter: Canbin Zheng
> Assignee: Canbin Zheng
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> So far, Flink has made efforts for the native integration of Kubernetes.
> However, it is always essential to evaluate the existing design and consider
> alternatives that have better design and are easier to maintain in the long
> run. We have suffered from some problems while developing new features base
> on the current code. Here is some of them:
> # We don’t have a unified monadic-step based orchestrator architecture to
> construct all the Kubernetes resources.
> ** There are inconsistencies between the orchestrator architecture that
> client uses to create the Kubernetes resources, and the orchestrator
> architecture that the master uses to create Pods; this confuses new
> contributors, as there is a cognitive burden to understand two architectural
> philosophies instead of one; for another, maintenance and new feature
> development become quite challenging.
> ** Pod construction is done in one step. With the introduction of new
> features for the Pod, the construction process could become far more
> complicated, and the functionality of a single class could explode, which
> hurts code readability, writability, and testability. At the moment, we have
> encountered such challenges and realized that it is not an easy thing to
> develop new features related to the Pod.
> ** The implementations of a specific feature are usually scattered in
> multiple decoration classes. For example, the current design uses a
> decoration class chain that contains five Decorator class to mount a
> configuration file to the Pod. If people would like to introduce other
> configuration files support, such as Hadoop configuration or Keytab files,
> they have no choice but to repeat the same tedious and scattered process.
> # We don’t have dedicated objects or tools for centrally parsing, verifying,
> and managing the Kubernetes parameters, which has raised some maintenance and
> inconsistency issues.
> ** There are many duplicated parsing and validating code, including settings
> of Image, ImagePullPolicy, ClusterID, ConfDir, Labels, etc. It not only harms
> readability and testability but also is prone to mistakes. Refer to issue
> FLINK-16025 for inconsistent parsing of the same parameter.
> ** The parameters are scattered so that some of the method signatures have
> to declare many unnecessary input parameters, such as
> FlinkMasterDeploymentDecorator#createJobManagerContainer.
>
> For solving these issues, we propose to
> # Introduce a unified monadic-step based orchestrator architecture that has
> a better, cleaner and consistent abstraction for the Kubernetes resources
> construction process.
> # Add some dedicated tools for centrally parsing, verifying, and managing
> the Kubernetes parameters.
>
> Refer to the design doc for the details, any feedback is welcome.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
