[
https://issues.apache.org/jira/browse/FLINK-16194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Canbin Zheng updated FLINK-16194:
---------------------------------
Summary: Refactor the Kubernetes decorator design (was: Refactor the
Kubernetes architecture design)
> Refactor the Kubernetes decorator design
> ----------------------------------------
>
> Key: FLINK-16194
> URL: https://issues.apache.org/jira/browse/FLINK-16194
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / Kubernetes
> Affects Versions: 1.10.0
> Reporter: Canbin Zheng
> Priority: Critical
> Fix For: 1.11.0
>
>
> So far, Flink has made efforts for the native integration of Kubernetes.
> However, it is always essential to evaluate the existing design and consider
> alternatives that have better design and are easier to maintain in the long
> run. We have suffered from some problems while developing new features base
> on the current code. Here is some of them:
> # We don’t have a unified monadic-step based orchestrator architecture to
> construct all the Kubernetes resources.
> ** There are inconsistencies between the orchestrator architecture that
> client uses to create the Kubernetes resources, and the orchestrator
> architecture that the master uses to create Pods; this confuses new
> contributors, as there is a cognitive burden to understand two architectural
> philosophies instead of one; for another, maintenance and new feature
> development become quite challenging.
> ** Pod construction is done in one step. With the introduction of new
> features for the Pod, the construction process could become far more
> complicated, and the functionality of a single class could explode, which
> hurts code readability, writability, and testability. At the moment, we have
> encountered such challenges and realized that it is not an easy thing to
> develop new features related to the Pod.
> ** The implementations of a specific feature are usually scattered in
> multiple decoration classes. For example, the current design uses a
> decoration class chain that contains five Decorator class to mount a
> configuration file to the Pod. If people would like to introduce other
> configuration files support, such as Hadoop configuration or Keytab files,
> they have no choice but to repeat the same tedious and scattered process.
> # We don’t have dedicated objects or tools for centrally parsing, verifying,
> and managing the Kubernetes parameters, which has raised some maintenance and
> inconsistency issues.
> ** There are many duplicated parsing and validating code, including settings
> of Image, ImagePullPolicy, ClusterID, ConfDir, Labels, etc. It not only harms
> readability and testability but also is prone to mistakes. Refer to issue
> FLINK-16025 for inconsistent parsing of the same parameter.
> ** The parameters are scattered so that some of the method signatures have
> to declare many unnecessary input parameters, such as
> FlinkMasterDeploymentDecorator#createJobManagerContainer.
>
> For solving these issues, we propose to
> # Introduce a unified monadic-step based orchestrator architecture that has
> a better, cleaner and consistent abstraction for the Kubernetes resources
> construction process.
> # Add some dedicated tools for centrally parsing, verifying, and managing
> the Kubernetes parameters.
>
> Refer to the design doc for the details, any feedback is welcome.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
