Hi guys,
Thanks for the discussion and great suggestions.
Here I summarize the functions to be completed in version 1.2.4
1.The table structure of the variable contains the following main contents
- variable code is placeholder, For example the variable code is
"collect.kafka.cluster", it will be referenced as ${collect.kafka.cluster},
it is unique within a team
- variable name is removed, after discussion, it is not necessarily retained
- variable value is the value of a specific variable, such as ip and port
- description is a detailed introduction
- team is the team the variable belongs to
2.Basic functions of variable management
- Add and modify variables: need to check whether the variable code is
unique under the current team
- Deleting a variable: It is necessary to check whether it has been
referenced by the Application under the team to which the variable belongs.
If there is metadata management in the future, it is also necessary to
check whether it has been referenced by the metadata. If it is referenced,
it will prompt that the variable is referenced and cannot be deleted.
3.References in Flink sql and program args
- In the application, the variables are uniformly referenced in the form of
placeholders ${xxx}, such as in Flink sql, program args
- For a better experience, it is necessary to provide the function of
following the search prompt when entering variable placeholders. For
example, when entering "${", the variable list is displayed, and when
"${kafka" is entered, the display contains variables for kafka strings, in
"variable code" or "description".
- When the focus is lost, it is judged whether there is a variable
placeholder, and if so, a verification button is displayed to prompt the
user to perform verification. When the user clicks the verification button,
the replaced content of the placeholder is displayed, and only the user
clicks the verification button to click the submit button
- If the user enters "okir23${j2Rts}pEt", this placeholder "${j2Rts}" will
not be replaced if "j2Rts" is not a variable we defined
4.Future plan
- Display the application or metadata list of a referenced variable in the
variable basic management module
- Improve user experience
Best,
Chunjin Mu
Chunjin Mu <[email protected]> 于2022年10月14日周五 22:28写道：
> When we have dozens or dozens of variables, this phenomenon may be more
> prominent
>
>
> Best,
> Chunjin Mu
>
> Chunjin Mu <[email protected]> 于2022年10月14日周五 22:26写道：
>
>> When a person asks which kafka should be used, I tell him in Chinese that
>> he should use "采集整合的kafka", in fact the variable code is collect.kafka, but
>> he is searching for gather keyword, it is difficult to just pass
>> collect.kafka The intuitive understanding is to "采集整合的kafka", because
>> the information contained in the variable code is still not that much.
>>
>> Huajie Wang <[email protected]> 于2022年10月14日周五 22:17写道：
>>
>>> I just thought about it, there is really no need for this name.
>>>
>>>
>>>
>>> Best,
>>> Huajie Wang
>>>
>>>
>>>
>>> Rui Fan <[email protected]> 于2022年10月14日周五 21:54写道：
>>>
>>> > Hi guys,
>>> >
>>> > I understand chunjin's solution is this:
>>> >
>>> > - variable code is placeholder
>>> > - variable name is an introduction
>>> > - Description is a detailed introduction
>>> >
>>> > I think introduction and detail are redundant.
>>> >
>>> > For example: there are two kafka clusters, the Beijing kafka
>>> > cluster and the Hangzhou kafka cluster.
>>> >
>>> > If placeholder is `beijing_kafka`, `hangzhou_kafka`. Why
>>> > does the introduction still need `北京 kafka 集群` and
>>> > `杭州 kafka 集群`? placeholder is already clear.
>>> >
>>> > If dev may forget the placeholder, then dev may also
>>> > forget the introduction.
>>> >
>>> > Of course, if the placeholder is kafka1 kafka2, an
>>> > introduction is required. Why do we need an introduction
>>> > at this point, because the placeholder name is unreasonable.
>>> >
>>> > Just like variable naming in Java code, there is no need to
>>> > introduce an introduction with a clear name. If the meaning of
>>> > the variable is complicated, we also need to write a detailed
>>> > description.
>>> >
>>> > Finally, I want to say: less is more. If we want to make
>>> > something complicated, please give everyone a strong reason.
>>> >
>>> > Best
>>> > fanrui
>>> >
>>> > On Fri, Oct 14, 2022 at 8:42 PM Huajie Wang <[email protected]> wrote:
>>> >
>>> > > Hi guys,
>>> > >
>>> > > I agree with Chunjin, I think variable name is useful, Chunjin also
>>> > > explained the reason.
>>> > >
>>> > >
>>> > > Best,
>>> > > Huajie Wang
>>> > >
>>> > >
>>> > >
>>> > > Chunjin Mu <[email protected]> 于2022年10月14日周五 20:22写道：
>>> > >
>>> > > > Hi, Rui Fan
>>> > > > I'll start a discussion later.
>>> > > >
>>> > > > Best
>>> > > > Chunjin Mu
>>> > > >
>>> > > > Rui Fan <[email protected]> 于2022年10月14日周五 19:42写道：
>>> > > >
>>> > > > > Hi guys,
>>> > > > >
>>> > > > > I agree with monster. And I suggest use variable name, variable
>>> value
>>> > > and
>>> > > > > description.
>>> > > > >
>>> > > > > By the way, there are some problems still need to confirm.
>>> What’s the
>>> > > > > placeholder? And how to use them when edit sql or datastream job?
>>> > It’s
>>> > > > > better to discuss these asap.
>>> > > > >
>>> > > > > Best
>>> > > > > fanrui
>>> > > > >
>>> > > > > On Fri, Oct 14, 2022 at 17:44 Chunjin Mu <[email protected]>
>>> > wrote:
>>> > > > >
>>> > > > > > hi, huajie, fan rui, monster
>>> > > > > >
>>> > > > > > Thank you for your suggestions.
>>> > > > > >
>>> > > > > > Now please Huajie to confirm how to do it. If remove the
>>> variable
>>> > > > name, I
>>> > > > > > can also accept it here. I will do small secondary development
>>> on
>>> > our
>>> > > > > > production system based on the community to meet internal
>>> needs.
>>> > > > > >
>>> > > > > > 陈卓宇 <[email protected]> 于2022年10月14日周五 15:58写道：
>>> > > > > >
>>> > > > > > > "外部采集整合kafka" is the Chinese description of variable, so it
>>> makes
>>> > > > sense
>>> > > > > > to
>>> > > > > > > put it under `Description`.
>>> > > > > > >
>>> > > > > > > Chunjin Mu <[email protected]> 于2022年10月14日周五 12:01写道：
>>> > > > > > >
>>> > > > > > > > The variable management module has three properties,
>>> including
>>> > > > > variable
>>> > > > > > > > code, variable name, and description, For example, the
>>> variable
>>> > > > code
>>> > > > > is
>>> > > > > > > > "external.kafka.collect.brokers", and the variable name is
>>> > > > > > "外部采集整合kafka".
>>> > > > > > > > When we created the application, I forgot what the variable
>>> > code
>>> > > > was,
>>> > > > > > > but I
>>> > > > > > > > knew it was called "采集整合", so I can search Enter the "采集整合"
>>> > > > keywords
>>> > > > > in
>>> > > > > > > the
>>> > > > > > > > box, and of course you can also search for kafka keywords.
>>> > > > > > > >
>>> > > > > > > > At the same time, I want to know more detailed information
>>> > about
>>> > > > this
>>> > > > > > > > kafka, such as who the operator is, what is the data
>>> retention
>>> > > > > period,
>>> > > > > > > and
>>> > > > > > > > how big is the cluster size, but I can't use the
>>> description to
>>> > > > > search,
>>> > > > > > > so
>>> > > > > > > > I can only search or variable code or variable name, and
>>> the
>>> > > > variable
>>> > > > > > > name
>>> > > > > > > > is also for a simple display.
>>> > > > > > > >
>>> > > > > > > > Although external.kafka.collect.brokers already contains a
>>> lot
>>> > of
>>> > > > > > > > information, I think this is just a naming convention.
>>> When we
>>> > > > > discuss
>>> > > > > > > this
>>> > > > > > > > variable, we will discuss it by using the "采集整合 kafka"
>>> > > > > > > >
>>> > > > > > > > We can discuss whether there is a problem with this design,
>>> > what
>>> > > do
>>> > > > > you
>>> > > > > > > > think?
>>> > > > > > > >
>>> > > > > > > > Rui Fan <[email protected]> 于2022年10月10日周一 11:45写道：
>>> > > > > > > >
>>> > > > > > > > > Hi huajie:
>>> > > > > > > > >
>>> > > > > > > > > > If it is manually entered, it will always be treated as
>>> > > > ordinary
>>> > > > > > > text.
>>> > > > > > > > >
>>> > > > > > > > > Users always copy some code from an old job to a new job.
>>> > > > > > > > > I'm afraid it might not meet the user's expectations.
>>> > > > > > > > >
>>> > > > > > > > > Best,
>>> > > > > > > > > fanrui
>>> > > > > > > > >
>>> > > > > > > > > On Sat, Oct 8, 2022 at 3:12 PM Huajie Wang <
>>> > [email protected]>
>>> > > > > > wrote:
>>> > > > > > > > >
>>> > > > > > > > > > hi 田向阳:
>>> > > > > > > > > >
>>> > > > > > > > > > The current discussion is about variable management,
>>> which
>>> > > is a
>>> > > > > > > > > relatively
>>> > > > > > > > > > independent function, No other modules involved, you
>>> can
>>> > > check
>>> > > > > the
>>> > > > > > > > thread
>>> > > > > > > > > > [1]
>>> > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > [1] thread:
>>> > > > > > > > > >
>>> > > > https://lists.apache.org/thread/s9sjpkjn6spvq07bk34cfd3vln7rjo0m
>>> > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > Best,
>>> > > > > > > > > > Huajie Wang
>>> > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > 田向阳 <[email protected]> 于2022年10月8日周六 13:13写道：
>>> > > > > > > > > >
>>> > > > > > > > > > > is this module include a resource management cente r
>>> > where
>>> > > > jars
>>> > > > > > can
>>> > > > > > > > be
>>> > > > > > > > > > > chosen to provide to create a job
>>> > > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > ---- 回复的原邮件 ----
>>> > > > > > > > > > > | 发件人 | Huajie Wang<[email protected]> |
>>> > > > > > > > > > > | 日期 | 2022年10月08日 12:07 |
>>> > > > > > > > > > > | 收件人 | [email protected]<
>>> > > [email protected]>
>>> > > > |
>>> > > > > > > > > > > | 抄送至 | |
>>> > > > > > > > > > > | 主题 | Re: [DISCUSS][issue-1477] Add variable
>>> management
>>> > > > > module |
>>> > > > > > > > > > > hi 阿穆:
>>> > > > > > > > > > > Thank you for the clarification, My question is
>>> answered,
>>> > > but
>>> > > > > > now I
>>> > > > > > > > > have
>>> > > > > > > > > > > one more concern,
>>> > > > > > > > > > > I'm currently using {var} for variables, and I'm
>>> worried
>>> > > that
>>> > > > > one
>>> > > > > > > of
>>> > > > > > > > > the
>>> > > > > > > > > > > user's parameters includes "{" and "}", e.g :
>>> > > > > > > > > > >
>>> > > > > > > > > > > --password 'mypass{123}abc'
>>> > > > > > > > > > >
>>> > > > > > > > > > > So how do we parse parse this? How about ${var}
>>> > > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > Best,
>>> > > > > > > > > > > Huajie Wang
>>> > > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > 阿穆 <[email protected]> 于2022年10月8日周六 11:36写道：
>>> > > > > > > > > > >
>>> > > > > > > > > > > > Hi, monster, Huajie Wang, Rui Fan:
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > Thanks for your reply.
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > I have integrated your replies. I really don't pay
>>> much
>>> > > > > > attention
>>> > > > > > > > to
>>> > > > > > > > > > SQL,
>>> > > > > > > > > > > > so I will do the following.
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > 1.Each team has independent variables that do not
>>> > affect
>>> > > > each
>>> > > > > > > other
>>> > > > > > > > > and
>>> > > > > > > > > > > > are isolated from each other, but the same
>>> variable is
>>> > > > unique
>>> > > > > > > under
>>> > > > > > > > > the
>>> > > > > > > > > > > > same team, so you need to specify the team when
>>> > creating
>>> > > > > > > variables
>>> > > > > > > > > > (users
>>> > > > > > > > > > > > may belong to multiple teams, and the teams of the
>>> > > current
>>> > > > > user
>>> > > > > > > > will
>>> > > > > > > > > be
>>> > > > > > > > > > > > displayed here for users to choose)
>>> > > > > > > > > > > > 2.For Flink jar jobs, there are two ways to pass
>>> > > variables
>>> > > > > > > > > > > > - By using placeholders, such as adding --kafka
>>> > {mykafka}
>>> > > > to
>>> > > > > > the
>>> > > > > > > > args
>>> > > > > > > > > > of
>>> > > > > > > > > > > > the application, 'mykafka' is the variable code of
>>> a
>>> > > > variable
>>> > > > > > > > > > > > - Search by variable code or variable name. After a
>>> > > > variable
>>> > > > > is
>>> > > > > > > > > > selected,
>>> > > > > > > > > > > > the parameter transfer method is displayed
>>> > > > > > > > > > > > 3.Flink sql will use variables as placeholders,
>>> E.g:
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > CREATE TABLE KafkaTableA (
>>> > > > > > > > > > > > `user_id` BIGINT,
>>> > > > > > > > > > > > `item_id` BIGINT,
>>> > > > > > > > > > > > `type` STRING
>>> > > > > > > > > > > > ) WITH (
>>> > > > > > > > > > > > 'connector' = 'kafka',
>>> > > > > > > > > > > > 'topic' = 'user_behaviorA',
>>> > > > > > > > > > > > 'properties.bootstrap.servers' =
>>> > > > > > > > > '{kafka_clusterA}',
>>> > > > > > > > > > > --
>>> > > > > > > > > > > > 'kafka_ ClusterA' is the variable code
>>> > > > > > > > > > > > 'properties.group.id' = 'testGroupA',
>>> > > > > > > > > > > > 'scan.startup.mode' =
>>> 'earliest-offset',
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > CREATE TABLE KafkaTableB (
>>> > > > > > > > > > > > `user_id` BIGINT,
>>> > > > > > > > > > > > `item_id` BIGINT
>>> > > > > > > > > > > > ) WITH (
>>> > > > > > > > > > > > 'connector' = 'kafka',
>>> > > > > > > > > > > > 'topic' = 'user_behaviorB',
>>> > > > > > > > > > > > 'properties.bootstrap.servers' =
>>> > > > > > > > > '{kafka_clusterB}',
>>> > > > > > > > > > > --
>>> > > > > > > > > > > > 'kafka_ ClusterB' is the variable code
>>> > > > > > > > > > > > 'properties.group.id' = 'testGroupB',
>>> > > > > > > > > > > > 'scan.startup.mode' =
>>> 'earliest-offset',
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > INSERT INTO KafkaTableA SELECT user_id, item_id,
>>> > '{type}'
>>> > > > > FROM
>>> > > > > > > > > > > > KafkaTableB -- 'type' is the variable
>>> code
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > 4.In the future, we can consider setting some
>>> system
>>> > > > built-in
>>> > > > > > > > > variables
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > Best,
>>> > > > > > > > > > > > ChunJin Mu
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > On Fri, Oct 7, 2022 at 8:27 PM Rui Fan <
>>> > > > [email protected]
>>> > > > > > >
>>> > > > > > > > > > wrote:
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > > Hi 阿穆:
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > It's a great proposal.
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > For usage scenarios, I strongly recommend
>>> support
>>> > > for
>>> > > > > > > > replacing
>>> > > > > > > > > > > > variables
>>> > > > > > > > > > > > > in sql. E.g:
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > CREATE TABLE KafkaTable (
>>> > > > > > > > > > > > > `user_id` BIGINT,
>>> > > > > > > > > > > > > `item_id` BIGINT
>>> > > > > > > > > > > > > ) WITH (
>>> > > > > > > > > > > > > 'connector' = 'kafka',
>>> > > > > > > > > > > > > 'topic' = 'user_behavior',
>>> > > > > > > > > > > > >
>>> 'properties.bootstrap.servers' =
>>> > > > > > > > > > > '{kafka_clusterA}',
>>> > > > > > > > > > > > > 'properties.group.id' =
>>> > > > 'testGroup',
>>> > > > > > > > > > > > > 'scan.startup.mode' =
>>> > > > > 'earliest-offset',
>>> > > > > > > > > > > > > 'format' = 'csv'
>>> > > > > > > > > > > > > )
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > Note: 'properties.bootstrap.servers' =
>>> > > > > '{kafka_clusterA}',
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > For variable management, I have some question:
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > 1. Is the parameter used within the team? If
>>> all
>>> > > teams
>>> > > > > > use a
>>> > > > > > > > > > common
>>> > > > > > > > > > > > > parameter, it
>>> > > > > > > > > > > > > needs to be defined for each team, right?
>>> > > > > > > > > > > > > 2. Which users can modify variables? Can all
>>> users
>>> > > > > modify
>>> > > > > > > all
>>> > > > > > > > > > > > variables of
>>> > > > > > > > > > > > > their team?
>>> > > > > > > > > > > > > After the variable is modified incorrectly,
>>> many
>>> > > jobs
>>> > > > > may
>>> > > > > > be
>>> > > > > > > > > > > > affected. So I
>>> > > > > > > > > > > > > suggest that
>>> > > > > > > > > > > > > only team admin or system admin have
>>> permission to
>>> > > > > modify,
>>> > > > > > > add
>>> > > > > > > > > and
>>> > > > > > > > > > > > delete
>>> > > > > > > > > > > > > variables.
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > Best
>>> > > > > > > > > > > > > fanrui
>>> > > > > > > > > > > >
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > On Fri, Oct 7, 2022 at 7:01 PM Huajie Wang <
>>> > > > > [email protected]
>>> > > > > > > >
>>> > > > > > > > > > wrote:
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > > hi 阿穆:
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > Thanks for starting a very useful discussion,
>>> I
>>> > > have a
>>> > > > > > some
>>> > > > > > > > > > > questions:
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > 1. The access boundary problem of variable
>>> > > management,
>>> > > > > is
>>> > > > > > > this
>>> > > > > > > > > all
>>> > > > > > > > > > > > > people in the platform can access these
>>> variables(
>>> > > no
>>> > > > > > matter
>>> > > > > > > > who
>>> > > > > > > > > > > > > defines) or that only team members can be
>>> access
>>> > > these
>>> > > > > > > > > variables.
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > 2. There are two usage scenarios for variables
>>> > that
>>> > > I
>>> > > > > know
>>> > > > > > > of,
>>> > > > > > > > > one
>>> > > > > > > > > > > is
>>> > > > > > > > > > > > > in the entry parameter of main, and the other
>>> is
>>> > in
>>> > > > > > > flinksql.
>>> > > > > > > > > Are
>>> > > > > > > > > > > both
>>> > > > > > > > > > > > > supported, or only supported main entry ?
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > about first question, my opinion:
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > 1. The variable management is associated with
>>> the
>>> > > > team.
>>> > > > > > > After
>>> > > > > > > > > the
>>> > > > > > > > > > > user
>>> > > > > > > > > > > > > logs in to the system, he must select a team
>>> (a
>>> > user
>>> > > > can
>>> > > > > > > > belong
>>> > > > > > > > > to
>>> > > > > > > > > > > one
>>> > > > > > > > > > > > > or more teams), The variable belongs to a team
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > 2. All resources between teams are isolated,
>>> which
>>> > > > means
>>> > > > > > > that
>>> > > > > > > > > > > > > variables are also isolated. In short,
>>> members of
>>> > a
>>> > > > team
>>> > > > > > can
>>> > > > > > > > use
>>> > > > > > > > > > and
>>> > > > > > > > > > > > > define these variables, which are invisible to
>>> > other
>>> > > > > > > members,
>>> > > > > > > > > and
>>> > > > > > > > > > > > > other members cannot use these variables.
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > Welcome to your views
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > Best,
>>> > > > > > > > > > > > > Huajie Wang
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > 陈卓宇 <[email protected]>
>>> > 于2022年10月7日周五
>>> > > > > > > 14:18写道：
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > > > Is it possible to do this variable
>>> support in
>>> > > sql
>>> > > > > as
>>> > > > > > > well
>>> > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > 阿穆 <[email protected]>
>>> > 于2022年10月7日周五
>>> > > > > > > 13:54写道：
>>> > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > Hi guys,
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > I want to start a discussion about
>>> the
>>> > > > > variable
>>> > > > > > > > > > management
>>> > > > > > > > > > > > module. Please
>>> > > > > > > > > > > > > > > see this issue issue-1477.
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > Background:
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > In the actual production
>>> environment,
>>> > > Flink
>>> > > > > jobs
>>> > > > > > > are
>>> > > > > > > > > > > > generally complex,
>>> > > > > > > > > > > > > > > and usually require multiple
>>> external
>>> > > > > > components.
>>> > > > > > > > For
>>> > > > > > > > > > > > example, Flink jobs
>>> > > > > > > > > > > > > > > consume data from Kafka, then
>>> connect
>>> > > > external
>>> > > > > > > > > > components
>>> > > > > > > > > > > > such as HBase
>>> > > > > > > > > > > > > > or
>>> > > > > > > > > > > > > > > Redis to obtain additional business
>>> > > > > information,
>>> > > > > > > and
>>> > > > > > > > > > then
>>> > > > > > > > > > > > write it to the
>>> > > > > > > > > > > > > > > downstream external components.
>>> There
>>> > are
>>> > > > the
>>> > > > > > > > > following
>>> > > > > > > > > > > > problems:
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > 1. The connection information of
>>> > external
>>> > > > > > > > components,
>>> > > > > > > > > > such
>>> > > > > > > > > > > > as IP, port
>>> > > > > > > > > > > > > > and
>>> > > > > > > > > > > > > > > user password, needs to be
>>> configured in
>>> > > the
>>> > > > > > > > > application
>>> > > > > > > > > > > > args and
>>> > > > > > > > > > > > > > > transferred to the Flink job, so
>>> that
>>> > the
>>> > > > > > > connection
>>> > > > > > > > > > > > information of
>>> > > > > > > > > > > > > > > external components is distributed
>>> in
>>> > > > multiple
>>> > > > > > > > > > > > applications. Once the
>>> > > > > > > > > > > > > > > connection information of external
>>> > > > components
>>> > > > > > > > changes,
>>> > > > > > > > > > > many
>>> > > > > > > > > > > > application
>>> > > > > > > > > > > > > > > args parameters need to be modified,
>>> > which
>>> > > > > will
>>> > > > > > > lead
>>> > > > > > > > > to
>>> > > > > > > > > > > > high operation
>>> > > > > > > > > > > > > > and
>>> > > > > > > > > > > > > > > maintenance costs.
>>> > > > > > > > > > > > > > > 2. There are many people in the team
>>> > > > > developing
>>> > > > > > > > > > real-time
>>> > > > > > > > > > > > computing jobs.
>>> > > > > > > > > > > > > > > There is no uniform specification
>>> for
>>> > the
>>> > > > > > > connection
>>> > > > > > > > > > > > information of the
>>> > > > > > > > > > > > > > > external components passed to the
>>> job,
>>> > > > > resulting
>>> > > > > > > in
>>> > > > > > > > > > > > different parameter
>>> > > > > > > > > > > > > > > names of the same component. This is
>>> > > > difficult
>>> > > > > > to
>>> > > > > > > > > count
>>> > > > > > > > > > > > which external
>>> > > > > > > > > > > > > > > components are dependent.
>>> > > > > > > > > > > > > > > 3. In production practice, there are
>>> > > usually
>>> > > > > > > > multiple
>>> > > > > > > > > > sets
>>> > > > > > > > > > > > of
>>> > > > > > > > > > > > > > > environments, such as test
>>> environment
>>> > and
>>> > > > > > > > production
>>> > > > > > > > > > > > environment. It is
>>> > > > > > > > > > > > > > > not intuitive to judge whether a
>>> > component
>>> > > > > > belongs
>>> > > > > > > > to
>>> > > > > > > > > a
>>> > > > > > > > > > > > test environment
>>> > > > > > > > > > > > > > or
>>> > > > > > > > > > > > > > > a production environment simply
>>> through
>>> > IP
>>> > > > and
>>> > > > > > > > ports.
>>> > > > > > > > > > Even
>>> > > > > > > > > > > > if it can be
>>> > > > > > > > > > > > > > > judged, there are some omissions. In
>>> > this
>>> > > > way,
>>> > > > > > the
>>> > > > > > > > > > > > connection information
>>> > > > > > > > > > > > > > > online to the production
>>> environment may
>>> > > be
>>> > > > > > > external
>>> > > > > > > > > > > > components of the
>>> > > > > > > > > > > > > > test
>>> > > > > > > > > > > > > > > environment, or components of the
>>> > > production
>>> > > > > > > > > environment
>>> > > > > > > > > > > are
>>> > > > > > > > > > > > > > inadvertently
>>> > > > > > > > > > > > > > > configured during testing, leading
>>> to
>>> > > > > production
>>> > > > > > > > > > failures.
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > Proposal:
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > 1. Create a unified module to manage
>>> > > > > variables.
>>> > > > > > > > Users
>>> > > > > > > > > > can
>>> > > > > > > > > > > > define their
>>> > > > > > > > > > > > > > own
>>> > > > > > > > > > > > > > > variables in this module. Variable
>>> > > > information
>>> > > > > > > > > includes
>>> > > > > > > > > > > > variable code
>>> > > > > > > > > > > > > > > (passed to Flink program as
>>> parameter
>>> > > name),
>>> > > > > > > > variable
>>> > > > > > > > > > > > value, variable
>>> > > > > > > > > > > > > name,
>>> > > > > > > > > > > > > > > variable description, and which
>>> > > applications
>>> > > > > > > depend
>>> > > > > > > > on
>>> > > > > > > > > > > > > > > 2. In the same Team module, the
>>> variable
>>> > > > code
>>> > > > > or
>>> > > > > > > > > > variable
>>> > > > > > > > > > > > name is unique
>>> > > > > > > > > > > > > > > - By using placeholders, such as
>>> adding
>>> > --
>>> > > > > kafka
>>> > > > > > > > > > {mykafka}
>>> > > > > > > > > > > > to the args of
>>> > > > > > > > > > > > > > > the application, mykafka is the
>>> variable
>>> > > > code
>>> > > > > > of a
>>> > > > > > > > > > > variable
>>> > > > > > > > > > > > > > > - Search by variable code or
>>> variable
>>> > > name.
>>> > > > > > After
>>> > > > > > > a
>>> > > > > > > > > > > > variable is selected,
>>> > > > > > > > > > > > > > > the parameter transfer method is
>>> > displayed
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > If you have any suggestions,
>>> welcome to
>>> > > > > > > participate
>>> > > > > > > > in
>>> > > > > > > > > > the
>>> > > > > > > > > > > > discussion on
>>> > > > > > > > > > > > > > > the mail list or issue, I look
>>> forward
>>> > to
>>> > > > your
>>> > > > > > > > > feedback.
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > Best wishes
>>> > > > > > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>
