Is it possible to do this variable support in sql as well
阿穆 <[email protected]> 于2022年10月7日周五 13:54写道：
> Hi guys,
>
>
> I want to start a discussion about the variable management module. Please
> see this issue issue-1477.
>
>
> Background:
>
>
> In the actual production environment, Flink jobs are generally complex,
> and usually require multiple external components. For example, Flink jobs
> consume data from Kafka, then connect external components such as HBase or
> Redis to obtain additional business information, and then write it to the
> downstream external components. There are the following problems:
>
>
> 1. The connection information of external components, such as IP, port and
> user password, needs to be configured in the application args and
> transferred to the Flink job, so that the connection information of
> external components is distributed in multiple applications. Once the
> connection information of external components changes, many application
> args parameters need to be modified, which will lead to high operation and
> maintenance costs.
> 2. There are many people in the team developing real-time computing jobs.
> There is no uniform specification for the connection information of the
> external components passed to the job, resulting in different parameter
> names of the same component. This is difficult to count which external
> components are dependent.
> 3. In production practice, there are usually multiple sets of
> environments, such as test environment and production environment. It is
> not intuitive to judge whether a component belongs to a test environment or
> a production environment simply through IP and ports. Even if it can be
> judged, there are some omissions. In this way, the connection information
> online to the production environment may be external components of the test
> environment, or components of the production environment are inadvertently
> configured during testing, leading to production failures.
>
>
> Proposal:
>
>
> 1. Create a unified module to manage variables. Users can define their own
> variables in this module. Variable information includes variable code
> (passed to Flink program as parameter name), variable value, variable name,
> variable description, and which applications depend on
> 2. In the same Team module, the variable code or variable name is unique
> - By using placeholders, such as adding -- kafka {mykafka} to the args of
> the application, mykafka is the variable code of a variable
> - Search by variable code or variable name. After a variable is selected,
> the parameter transfer method is displayed
>
>
>
> If you have any suggestions, welcome to participate in the discussion on
> the mail list or issue, I look forward to your feedback.
>
>
> Best wishes
