This is an automated email from the ASF dual-hosted git repository.
peacewong pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/linkis-website.git
The following commit(s) were added to refs/heads/dev by this push:
new e715ca13f4 update variables docs (#682)
e715ca13f4 is described below
commit e715ca13f48a53c8a96abf4f01d2d8af0e592f62
Author: peacewong <[email protected]>
AuthorDate: Thu Mar 2 14:30:07 2023 +0800
update variables docs (#682)
* update dynamic variables docs
* update add committer doc
---
community/how-to-vote-a-committer-pmc.md | 26 +-
docs/architecture/feature/commons/variable.md | 152 ++++-----
docs/user-guide/dynamic-variables.md | 359 +++++++++++++------
.../current/how-to-vote-a-committer-pmc.md | 27 +-
.../architecture/feature/commons/variable.md | 97 +++---
.../current/user-guide/dynamic-variables.md | 192 ++++++++++-
.../version-1.3.0/user-guide/dynamic-variables.md | 192 ++++++++++-
.../version-1.3.1/user-guide/dynamic-variables.md | 192 ++++++++++-
static/Images/Architecture/Commons/var_time.png | Bin 0 -> 30886 bytes
.../version-1.3.0/user-guide/dynamic-variables.md | 380 +++++++++++++++------
.../version-1.3.1/user-guide/dynamic-variables.md | 359 +++++++++++++------
11 files changed, 1441 insertions(+), 535 deletions(-)
diff --git a/community/how-to-vote-a-committer-pmc.md
b/community/how-to-vote-a-committer-pmc.md
index 607468ee7a..032e4784e0 100644
--- a/community/how-to-vote-a-committer-pmc.md
+++ b/community/how-to-vote-a-committer-pmc.md
@@ -276,31 +276,7 @@ The Apache Linkis PMC
At this point, the whole process is over, and the candidate officially becomes
the Committer or PMC of the project.
-## 4 Update project status
-step1 clong status record file
-```shell script
-svn co https://svn.apache.org/repos/asf/public/trunk/content/projects/
-```
-
-step2 update news infomation in linkis.xml
-add committer information record
-```shell script
-<section id="News">
- <title>News</title>
- <ul>
-<!-- <li>YYYY-MM-DD New committer: Fred Hampton</li> -->
- <li>2021-08-02 Project enters incubation.</li>
- <li>2022-02-19 First Apache Linkis release v1.0.3</li>
- <li>2022-02-24 New Committer: Chen Xia</li>
- <li>2022-04-15 Apache Linkis release v1.1.0</li>
-
- </ul>
- </section>
-```
-step3 the information show in follow file after update
-https://incubator.apache.org/projects/linkis.html
-
-## 5 Workflow summary
+## 4 Workflow summary
1. Send emails with ICLA attachments
diff --git a/docs/architecture/feature/commons/variable.md
b/docs/architecture/feature/commons/variable.md
index 81cea7b832..a2708c32dc 100644
--- a/docs/architecture/feature/commons/variable.md
+++ b/docs/architecture/feature/commons/variable.md
@@ -3,102 +3,92 @@ title: Custom Variable Design
sidebar_position: 1
---
-## 1. General
-### 1.1 Requirements Background
- Users want to be able to define some common
variables when writing code and then replace them during execution. For
example, users run the same sql in batches every day, and need to specify the
partition time of the previous day. If based on sql It will be more complicated
to write if the system provides a variable of run_date which will be very
convenient to use.
-### 1.2 Target
-1. Support variable substitution of task code
+## 1. Overview
+### need
+1. The user hopes that Linkis can provide some public variables and then
replace them during execution. For example, the user runs the same SQL in
batches every day, and needs to specify the partition time of the previous day.
Writing based on SQL will be more complicated if the system provides a run_date
variable It will be very convenient to use.
+2. The user hopes that Linkis supports date pattern calculation, supports
writing variables such as &{YYYY-MM-DD} in the code to calculate time variables
+3. The user wants to define variables by himself, such as setting a float
variable, and then use it in the code
+
+### Target
+1. Support variable replacement of task code
2. Support custom variables, support users to define custom variables in
scripts and task parameters submitted to Linkis, support simple +, - and other
calculations
3. Preset system variables: run_date, run_month, run_today and other system
variables
+4. Support date pattern variable, support +, - operation of pattern
## 2. Overall Design
- During the execution of the Linkis task, the
custom variables are carried out in Entrance, mainly through the interceptor of
Entrance before the task is submitted and executed. The variable and the
defined variable, and complete the code replacement through the initial value
of the custom variable passed in by the task, and become the final executable
code.
+ During the execution of the Linkis task, the
custom variable is carried out in Entrance, mainly through the interceptor of
Entrance before the task is submitted and executed. Variables and defined
variables, and the initial value of the custom variable passed in through the
task completes the code replacement and becomes the final executable code.
### 2.1 Technical Architecture
- The overall structure of custom variables is as
follows. After the task is submitted, it will go through the variable
replacement interceptor. First, all variables and expressions used in the code
will be parsed, and then replaced with the system and user-defined initial
values of variables, and finally the parsed code will be submitted to
EngineConn for execution. So the underlying engine is already replaced code.
+ The overall structure of custom variables is as
follows. After the task is submitted, it will pass through the variable
replacement interceptor. First, it will analyze all the variables and
expressions used in the code, and then replace them with the initial values of
the system and user-defined variables, and finally submit the parsed code to
EngineConn for execution. So the underlying engine is already replaced code.
+
+
-
+Remarks: Because the functions of variable and parsing are more general, the
extraction tool class is defined in linkis-commons:
org.apache.linkis.common.utils.VariableUtils
-## 3.Function introduction
- The variable types supported by Linkis are
divided into custom variables and system built-in variables. The internal
variables are predefined by Linkis and can be used directly. Then different
variable types support different calculation formats: String supports +,
integer decimal supports +-*/, date supports +-.
+### 2.2 Business Architecture
+ The feature this time is mainly to complete the
analysis, calculation, and replacement functions of variable substitution,
which mainly involves the Entrance module of linkis for code interception and
the variable substitution tools defined by the Linkis-commons module :
-### 3.1 Built-in variables
-The currently supported built-in variables are as follows:
+| Component Name | Level 1 Module | Level 2 Module | Function Point |
+|---|---|---|---|
+| Linkis | CG | Entrance|Intercept task code and call Linkis-common's
VariableUtils for code replacement|
+| Linkis | Linkis-commons | linkis-common|Provide variable, analysis,
calculation tool class VariableUtils|
-| variable name | variable type | variable meaning | variable value example |
-| ------ | -------- | -------- | ------------ |
-| run\_date | String | Data statistics time (support user's own setting, the
default setting is the day before the current time), if the data of yesterday
is executed today, it will be the time of yesterday, the format is yyyyMMdd |
20180129 |
-| run\_date\_std | String | Data statistics time (standard date format), if
yesterday's data is executed today, it will be yesterday's time, the format is
yyyy-MM-dd | 2018-01-29 |
-| run_today | String | The day after run_date (data statistics time), the
format is yyyyMMdd | 20211210 |
-| run_today_std | String | The day after run_date (data statistics time)
(standard format), the format is yyyy-MM-dd | 2021-12-10 |
-| run_mon | String | The month of the data statistics time, the format is
yyyyMM | 202112 |
-| run_mon_std | String | The month of the data statistics time (standard
format), the format is yyyy-MM | 2021-12 |
-| run\_month\_begin | String | The first day of the month in which the data is
counted, in the format yyyyMMdd | 20180101 |
-| run\_month\_begin\_std | String | The first day of the month where the data
statistics time is (standard date format), the format is yyyy-MM-dd |
2018-01-01 |
-| run_month_now_begin | String | The first day of the month where run_today is
in the format yyyyMMdd | 20211201 |
-| run_month_now_begin_std | String | The first day of the month run_today
(standard format), the format is yyyy-MM-dd | 2021-12-01 |
-| run\_month\_end | String | The last day of the month in which the data is
counted, in the format yyyyMMdd | 20180131 |
-| run\_month\_end\_std | String | The last day of the month in which the data
is counted (standard date format), the format is yyyy-MM-dd | 2018-01-31 |
-| run_month_now_end | String | The last day of the month where run_today is in
the format yyyyMMdd | 20211231 |
-| run_month_now_end_std | String | The last day of the month in which
run_today is located (standard date format), the format is yyyy-MM-dd |
2021-12-31 |
-| run_quarter_begin | String | The first day of the quarter in which the data
is counted, in the format yyyyMMdd | 20210401 |
-| run_quarter_end | String | The last day of the quarter in which the data is
counted, in the format yyyyMMdd | 20210630 |
-| run_half_year_begin | String | The first day of the half year where the data
statistics time is located, in the format yyyyMMdd | 20210101 |
-| run_half_year_end | String | The last day of the half year where the data
statistics time is located, the format is yyyyMMdd | 20210630 |
-| run_year_begin | String | The first day of the year in which the data is
counted, in the format yyyyMMdd | 20210101 |
-| run_year_end | String | The last day of the year in which the data is
counted, in the format yyyyMMdd | 20211231 |
-| run_quarter_begin_std | String | The first day of the quarter in which the
data is counted (standard format), the format is yyyy-MM-dd | 2021-10-01 |
-| run_quarter_end_std | String | The last day of the quarter where the data
statistics time is located (standard format), the format is yyyy-MM-dd |
2021-12-31 |
-| run_half_year_begin_std | String | The first day of the half year where the
data statistics time is located (standard format), the format is yyyy-MM-dd |
2021-07-01 |
-| run_half_year_end_std | String | The last day of the half year where the
data statistics time is located (standard format), the format is yyyy-MM-dd |
2021-12-31 |
-| run_year_begin_std | String | The first day of the year in which the data is
counted (standard format), the format is yyyy-MM-dd | 2021-01-01 |
-| run_year_end_std | String | The last day of the year in which the data is
counted (standard format), the format is yyyy-MM-dd | 2021-12-31 |
+## 3. Module design
+### 3.1 Core Execution Process
+[input port] The input port is code and code type (python/sql/scala/sh).
+[Processing flow] Entrance will first enter the interceptor after receiving
the task, and start the variable interceptor to complete the analysis,
replacement and calculation of variables
+The overall timing diagram is as follows:
-details:
+
-1. run_date is the core built-in date variable, which supports user-defined
date. If not specified, the default is the day before the current system time.
-2. Definition of other derived built-in date variables: other date built-in
variables are calculated relative to run_date. Once run_date changes, other
variable values will also change automatically. Other date variables do not
support setting initial values and can only be modified by modifying
run_date. .
-3. Built-in variables support more abundant usage scenarios: ${run_date-1} is
the day before run_data; ${run_month_begin-1} is the first day of the previous
month of run_month_begin, where -1 means minus one month.
+What needs to be explained here is:
+1. Custom variables and system variables are used in ${}, such as ${run_date}
+2. The date pattern variable is used as &{}, for example, the value of
&{yyyy-01-01} is 2022-01-01.
+ The reason why it is divided into two different ways is to prevent the
string defined by the custom variable from containing pattern characters. For
example, if a custom variable with y=1 is defined, it may represent different
meanings, and it will be the year variable by the pattern task.
-### 3.2 Custom variables
- What are custom variables? User variables that
are defined first and then used. User-defined variables temporarily support the
definition of strings, integers, and floating-point variables. Strings support
the + method, and integers and floating-point numbers support the +-*/ method.
User-defined variables do not conflict with the set variable syntax supported
by SparkSQL and HQL, but the same name is not allowed. How to define and use
custom variables? as follows:
-````
-## Defined in the code, specified before the task code
-sql type definition method:
---@set f=20.1
-The python/shell types are defined as follows:
-#@set f=20.1
-Note: Only one variable can be defined on one line
-````
-The use is directly used in the code through ```{varName expression}, such as
${f*2}```
+
+### 3.2 Specific details:
+1. run_date is a date variable that comes with the core, and supports
user-defined dates. If not specified, it defaults to the day before the current
system time.
+2. Definition of other derived built-in date variables: other date built-in
variables are calculated relative to run_date. Once run_date changes, the
values of other variables will also change automatically. Other date variables
do not support setting initial values and can only be modified by modifying
run_date .
+3. The built-in variables support richer usage scenarios: ${run_date-1} is the
day before run_data; ${run_month_begin-1} is the first day of the previous
month of run_month_begin, where -1 means minus one month.
+4. Pattern type variables are also calculated based on run_date, and then
replaced and +—
### 3.3 Variable scope
-Custom variables in linkis also have scope, and the priority is that the
variable defined in the script is greater than the Variable defined in the task
parameter is greater than the built-in run_date variable. The task parameters
are defined as follows:
-````
-##restful
+Custom variables also have a scope in linkis, and the priority is that the
variable defined in the script is greater than the Variable defined in the task
parameter and greater than the built-in run_date variable. The task parameters
are defined as follows:
+```
+## restful
{
- "executionContent": {"code": "select \"${f-1}\";", "runType": "sql"},
- "params": {
- "variable": {f: "20.1"},
- "configuration": {
- "runtime": {
-
"linkis.openlookeng.url":"http://127.0.0.1:9090";
- }
- }
- },
- "source": {"scriptPath": "file:///mnt/bdp/hadoop/1.sql"},
- "labels": {
- "engineType": "spark-2.4.3",
- "userCreator": "hadoop-IDE"
- }
+ "executionContent": {"code": "select \"${f-1}\";", "runType": "sql"},
+ "params": {
+ "variable": {f: "20.1"},
+ "configuration": {
+ "runtime": {
+
"linkis.openlookeng.url":"http://127.0.0.1:9090";
+ }
+ }
+ },
+ "source": {"scriptPath": "file:///mnt/bdp/hadoop/1.sql"},
+ "labels": {
+ "engineType": "spark-2.4.3",
+ "userCreator": "hadoop-IDE"
+ }
}
## java SDK
-JobSubmitAction.builder
- .addExecuteCode(code)
- .setStartupParams(startupMap)
- .setUser(user) //submit user
- .addExecuteUser(user) //execute user
- .setLabels(labels)
- .setVariableMap(varMap) //setVar
- .build
-````
\ No newline at end of file
+JobSubmitAction. builder
+ .addExecuteCode(code)
+ .setStartupParams(startupMap)
+ .setUser(user) //submit user
+ .addExecuteUser(user) //execute user
+ .setLabels(labels)
+ .setVariableMap(varMap) //setVar
+ .build
+```
+
+## 4. Interface design:
+The main tools are:
+```
+VariableUtils:
+def replace(replaceStr: String): String replaces the variable in the code and
returns the replaced code
+def replace(replaceStr: String, variables: util.Map[String, Any]): String
supports passing in the value of a custom variable for replacement
+def replace(code: String, runtType: String, variables: util.Map[String,
String]): String supports incoming code types, and performs replacement parsing
according to different types
+```
\ No newline at end of file
diff --git a/docs/user-guide/dynamic-variables.md
b/docs/user-guide/dynamic-variables.md
index c901317e1e..77b2d8b5fb 100644
--- a/docs/user-guide/dynamic-variables.md
+++ b/docs/user-guide/dynamic-variables.md
@@ -1,15 +1,184 @@
---
-title: Built In Time Variable
-sidebar_position: 7
+title: built-in time variable
+sidebar_position: 6
---
-## 1. General
-### 1.1 Requirements Background
-Users hope that when writing code, the time format requirements are
ever-changing, and the existing [Linkis custom
variables](https://linkis.apache.org/docs/latest/architecture/commons/variable/)
is currently not enough to support these requirements. In addition, some of
the existing time operation -1 means minus one month, and some minus one day,
which is easy for users to confuse
-### 1.2 Target
-* Other date built-in variables are calculated relative to run_date
-* Support Pattern format time and users can specify at will
+## 1. Overview
+### need
+1. The user hopes that Linkis can provide some public variables and then
replace them during execution. For example, the user runs the same SQL in
batches every day, and needs to specify the partition time of the previous day.
Writing based on SQL will be more complicated if the system provides a run_date
variable It will be very convenient to use.
+2. The user hopes that Linkis supports date pattern calculation, supports
writing variables such as &{YYYY-MM-DD} in the code to calculate time variables
+3. The user wants to define variables by himself, such as setting a float
variable, and then use it in the code
+
+### Target
+1. Support variable replacement of task code
+2. Support custom variables, support users to define custom variables in
scripts and task parameters submitted to Linkis, support simple +, - and other
calculations
+3. Preset system variables: run_date, run_month, run_today and other system
variables
+4. Support date pattern variable, support +, - operation of pattern
+
+The specific technical architecture can refer to:
+[Linkis Custom
Variables](https://linkis.apache.org/docs/latest/architecture/commons/variable/)
+
+## 2 Custom variable function introduction
+ The variable types supported by Linkis are
divided into custom variables and system built-in variables. The internal
variables are pre-defined by Linkis and can be used directly. Then different
variable types support different calculation formats: String supports +,
integers and decimals support +-*/, and dates support +-.
+
+### 2.1 Built-in variables
+Currently supported built-in variables are as follows:
+(The variable values in the following table take 20240229 in leap year as a
special date as an example)
+
+| variable name | variable type | variable meaning | variable value example |
+| ------ | -------- | -------- | ---------- |
+| run\_date | String | Data statistics time (users can set it themselves, the
default setting is the day before the current time), if you execute yesterday’s
data today, it will be yesterday’s time, the format is yyyyMMdd | 20240229 |
+| run\_date\_std | String | Data statistics time (standard date format), if
yesterday’s data is executed today, it is yesterday’s time, the format is
yyyy-MM-dd | 2024-02-29 |
+| run_today | String | The day after run_date (data statistics time), the
format is yyyyMMdd | 20240301 |
+| run_today_std | String | The day after run_date (data statistics time)
(standard format), the format is yyyy-MM-dd | 2024-03-01 |
+| run_mon | String | The month of run_date, the format is yyyyMM | 202402 |
+| run_mon_std | String | The month of run_date (standard format), the format
is yyyy-MM | 2024-02 |
+| run\_month\_begin | String | The first day of the month where the data
statistics time is located, the format is yyyyMMdd | 20240201 |
+| run\_month\_begin\_std | String | The first day of the month where run_date
is located (standard date format), the format is yyyy-MM-dd | 2024-02-01 |
+| run_month_now_begin | String | the first day of the previous month where
run_today is located, the format is yyyyMMdd | 20240201 |
+| run_month_now_begin_std | String | The first day of the previous month where
run_today is located (standard format), the format is yyyy-MM-dd | 2024-02-01 |
+| run\_month\_end | String | The last day of the month where run_date belongs,
in the format of yyyyMMdd | 20240229 |
+| run\_month\_end\_std | String | The last day of the month of run_date
(standard date format), the format is yyyy-MM-dd | 2024-02-29 |
+| run_month_now_end | String | the last day of the previous month where
run_today is located, the format is yyyyMMdd | 20240229 |
+| run_month_now_end_std | String | The last day of the previous month where
run_today is located (standard date format), the format is yyyy-MM-dd |
2024-02-29 |
+| run_quarter_begin | String | The first day of the quarter where run_date
belongs, the format is yyyyMMdd | 20240101 |
+| run_quarter_end | String | The last day of the quarter where run_date is
located, in the format of yyyyMMdd | 20240331 |
+| run_half_year_begin | String | The first day of the half year where run_date
is located, the format is yyyyMMdd | 20240101 |
+| run_half_year_end | String | The last day of the half year where run_date is
located, the format is yyyyMMdd | 20240630 |
+| run_year_begin | String | The first day of the year where run_date is
located, the format is yyyyMMdd | 20240101 |
+| run_year_end | String | The last day of the year where run_date is located,
the format is yyyyMMdd | 20241231 |
+| run_quarter_begin_std | String | The first day of the quarter where run_date
belongs (standard format), the format is yyyy-MM-dd | 2024-01-01 |
+| run_quarter_end_std | String | The last day of the quarter where run_date
belongs (standard format), the format is yyyy-MM-dd | 2024-03-31 |
+| run_half_year_begin_std | String | The first day of the half year where
run_date is located (standard format), the format is yyyy-MM-dd | 2024-01-01 |
+| run_half_year_end_std | String | The last day of the half year where
run_date is located (standard format), the format is yyyy-MM-dd | 2024-06-30 |
+| run_year_begin_std | String | The first day of the year where run_date is
located (standard format), the format is yyyy-MM-dd | 2024-01-01 |
+| run_year_end_std | String | The last day of the year of run_date (standard
format), the format is yyyy-MM-dd | 2024-12-31 |
+| run_tody_h | String | run_today task running time, yyyyMMddHH | 2024030111 |
+| run_tody_h_std | String | The running time of the run_today task yyyy-MM-dd
HH | 2024-03-01 11 |
+
+details:
+
+1. run_date is a date variable that comes with the core, and supports
user-defined dates. If not specified, it defaults to the day before the current
system time.
+2. Definition of other derived built-in date variables: other date built-in
variables are calculated relative to run_date. Once run_date changes, the
values of other variables will also change automatically. Other date variables
do not support setting initial values and can only be modified by modifying
run_date .
+3. The built-in variables support richer usage scenarios: ${run_date-1} is the
day before run_data; ${run_month_begin-1} is the first day of the previous
month of run_month_begin, where -1 means minus one month.
+
+Take sql as an example:
+```
+--@set run_date=20240229
+select
+"${run_date}" as run_date,
+"${run_date_std}" as run_date_std,
+"${run_today}" as run_today,
+"${run_today_std}" as run_today_std,
+"${run_mon}" as run_mon,
+"${run_mon_std}" as run_mon_std,
+"${run_month_begin}" as run_month_begin,
+"${run_month_begin_std}" as run_month_begin_std,
+"${run_month_now_begin}" as run_month_now_begin,
+"${run_month_now_begin_std}" as run_month_now_begin_std,
+"${run_month_end}" as run_month_end,
+"${run_month_end_std}" as run_month_end_std,
+"${run_month_now_end}" as run_month_now_end,
+"${run_month_now_end_std}" as run_month_now_end_std,
+"${run_quarter_begin}" as run_quarter_begin,
+"${run_quarter_end}" as run_quarter_end,
+"${run_half_year_begin}" as run_half_year_begin,
+"${run_half_year_end}" as run_half_year_end,
+"${run_year_begin}" as run_year_begin,
+"${run_year_end}" as run_year_end,
+"${run_quarter_begin_std}" as run_quarter_begin_std,
+"${run_quarter_end_std}" as run_quarter_end_std,
+"${run_half_year_begin_std}" as run_half_year_begin_std,
+"${run_half_year_end_std}" as run_half_year_end_std,
+"${run_year_begin_std}" as run_year_begin_std,
+"${run_year_end_std}" as run_year_end_std,
+"${run_today_h}" as run_tody_h,
+"${run_today_h_std}" as run_tody_h_std
+
+Compile the replaced content:
+select
+"20240229" as run_date,
+"2024-02-29" as run_date_std,
+"20240301" as run_today,
+"2024-03-01" as run_today_std,
+"202402" as run_mon,
+"2024-02" as run_mon_std,
+"20240201" as run_month_begin,
+"2024-02-01" as run_month_begin_std,
+"20240201" as run_month_now_begin,
+"2024-02-01" as run_month_now_begin_std,
+"20240229" as run_month_end,
+"2024-02-29" as run_month_end_std,
+"20240229" as run_month_now_end,
+"2024-02-29" as run_month_now_end_std,
+"20240101" as run_quarter_begin,
+"20240331" as run_quarter_end,
+"20240101" as run_half_year_begin,
+"20240630" as run_half_year_end,
+"20240101" as run_year_begin,
+"20241231" as run_year_end,
+"2024-01-01" as run_quarter_begin_std,
+"2024-03-31" as run_quarter_end_std,
+"2024-01-01" as run_half_year_begin_std,
+"2024-06-30" as run_half_year_end_std,
+"2024-01-01" as run_year_begin_std,
+"2024-12-31" as run_year_end_std,
+"2024030111" as run_tody_h,
+"2024-03-01 11" as run_tody_h_std
+
+```
+
+### 2.2 Custom variables
+ What are custom variables? User variables that
are defined first and used later. User-defined variables temporarily support
the definition of strings, integers, and floating-point variables. Strings
support the + method, and integers and floating-point numbers support the +-*/
method. User-defined variables do not conflict with the set variable syntax
supported by SparkSQL and HQL itself, but the same name is not allowed. How to
define and use custom variabl [...]
+```
+## Defined in the code, specify before the task code
+sql type definition method:
+--@set f=20.1
+The python/Shell type is defined as follows:
+#@set f=20.1
+Note: Only one line to define a variable is supported
+```
+The use is directly used in the code through ```{varName expression}, such as
${f*2}```
+
+### 2.3 Variable scope
+Custom variables also have a scope in linkis, and the priority is that the
variable defined in the script is greater than the Variable defined in the task
parameter and greater than the built-in run_date variable. The task parameters
are defined as follows:
+```
+## restful
+{
+ "executionContent": {"code": "select \"${f-1}\";", "runType": "sql"},
+ "params": {
+ "variable": {f: "20.1"},
+ "configuration": {
+ "runtime": {
+
"linkis.openlookeng.url":"http://127.0.0.1:9090";
+ }
+ }
+ },
+ "source": {"scriptPath": "file:///mnt/bdp/hadoop/1.sql"},
+ "labels": {
+ "engineType": "spark-2.4.3",
+ "userCreator": "hadoop-IDE"
+ }
+}
+## java SDK
+JobSubmitAction. builder
+ .addExecuteCode(code)
+ .setStartupParams(startupMap)
+ .setUser(user) //submit user
+ .addExecuteUser(user) //execute user
+ .setLabels(labels)
+ .setVariableMap(varMap) //setVar
+ .build
+```
+
+## 3. Date Pattern variable introduction
+
+### 3.1 Features
+
+* Support Pattern format time and users can specify it at will
+* Pattern date variables are calculated relative to run_date
* Support ±y/±M/±d/±H etc.
+* Among them, +- is to operate on the linkis built-in parameter run_date
first, and then replace the pattern field before %. Non-Pattern characters do
not support operation and replacement.
Pattern format comparison table:
@@ -25,7 +194,7 @@ D |Day in year |Number| 189
d |Day in month |Number |10
F |Day of week in month| Number |2
E |Day name in week |Text |Tuesday; Tue
-e |Day number of week (1 = Monday, …, 7 = Sunday) |Number |1
+u |Day number of week (1 = Monday, …, 7 = Sunday) |Number |1
a |Am/pm marker |Text |PM
H |Hour in day (0-23) |Number| 0
k |Hour in day (1-24)| Number |24
@@ -38,15 +207,9 @@ z |Time zone |General time zone |Pacific Standard Time;
PST; GMT-08:00
Z |Time zone |RFC 822 time zone |-0800
X |Time zone |ISO 8601 time zone |-08; -0800; -08:00
-## 2. Overall Design
-The overall design and technical architecture refer to [Linkis Custom
Variables](https://linkis.apache.org/docs/latest/architecture/commons/variable/)
-
-## 3. Function introduction
-* The variable types supported by Linkis are divided into custom variables
(not to be repeated) and system built-in variables. The custom variable date
supports +-.
-* Among them, +- is to perform operation on the built-in parameter run_date of
linkis, and then replace the pattern field before %. Non-pattern characters do
not support operation replacement.
-### 3.1 Examples of built-in variables
-You can define parameters that need to be dynamically rendered according to
your own preferences/business actual situation
+### 3.2 Date Pattern variable example
+You can define the parameters that need dynamic rendering according to your
own preferences/business actual situation
variable | result
--- | ---
@@ -59,7 +222,7 @@ variable | result
&{yyyyMM01%-1M} | 20210301
&{HH%-1H} | 14
-### 3.2 Custom Variable Usage Example
+### 3.3 Date Pattern variable usage examples
* Example 1: sql
```sql
@@ -84,96 +247,96 @@ aws s3 ls
s3://***/ads/tmp/dws_member_active_detail_d_20210601_20211231/pt=20220
* Example 3: datax json
```json
{
- "job": {
- "setting": {
- "speed": {
- "channel": 1
- }
- },
- "content": [
- {
- "reader": {
- "name": "s3reader",
- "parameter": {
- "bucket": "****************",
- "path": [
- "ads/tmp/ccass_tm_announcements/&{yyyyMMdd%-1d}/"
- ],
- "stored": "parquet",
- "compression": "NONE",
- "column": [
- {
- "index": 0,
- "type": "int"
- },
- {
- "index": 1,
- "type": "string",
- "constant": "&{yyyyMMdd%-1d}"
- }
- ]
- }
- },
- "writer": {
- "name": "streamwriter",
- "parameter": {
- "print": true
- }
- }
- }
- ]
- }
+ "job": {
+ "setting": {
+ "speed": {
+ "channel": 1
+ }
+ },
+ "content": [
+ {
+ "reader": {
+ "name": "s3reader",
+ "parameter": {
+ "bucket": "**************",
+ "path": [
+ "ads/tmp/ccass_tm_announcements/&{yyyyMMdd%-1d}/"
+ ],
+ "stored": "parquet",
+ "compression": "NONE",
+ "column": [
+ {
+ "index": 0,
+ "type": "int"
+ },
+ {
+ "index": 1,
+ "type": "string",
+ "constant": "&{yyyyMMdd%-1d}"
+ }
+ ]
+ }
+ },
+ "writer": {
+ "name": "streamwriter",
+ "parameter": {
+ "print": true
+ }
+ }
+ }
+ ]
+ }
}
```
after rendering
```json
{
- "job": {
- "setting": {
- "speed": {
- "channel": 1
- }
- },
- "content": [
- {
- "reader": {
- "name": "s3reader",
- "parameter": {
- "bucket": "****************",
- "path": [
- "ads/tmp/ccass_tm_announcements/20220705/"
- ],
- "stored": "parquet",
- "compression": "NONE",
- "column": [
- {
- "index": 0,
- "type": "int"
- },
- {
- "index": 1,
- "type": "string",
- "constant": "20220705"
- }
- ]
- }
- },
- "writer": {
- "name": "streamwriter",
- "parameter": {
- "print": true
- }
- }
- }
- ]
- }
+ "job": {
+ "setting": {
+ "speed": {
+ "channel": 1
+ }
+ },
+ "content": [
+ {
+ "reader": {
+ "name": "s3reader",
+ "parameter": {
+ "bucket": "**************",
+ "path": [
+ "ads/tmp/ccass_tm_announcements/20220705/"
+ ],
+ "stored": "parquet",
+ "compression": "NONE",
+ "column": [
+ {
+ "index": 0,
+ "type": "int"
+ },
+ {
+ "index": 1,
+ "type": "string",
+ "constant": "20220705"
+ }
+ ]
+ }
+ },
+ "writer": {
+ "name": "streamwriter",
+ "parameter": {
+ "print": true
+ }
+ }
+ }
+ ]
+ }
}
```
-* Example 4: python
+* Example 4:python
```python
print(&{yyyyMMdd%-1d})
```
after rendering
```
- 20220705
+ 20220705
```
\ No newline at end of file
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/how-to-vote-a-committer-pmc.md
b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/how-to-vote-a-committer-pmc.md
index 250183cd91..b526b6bb2c 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/how-to-vote-a-committer-pmc.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/how-to-vote-a-committer-pmc.md
@@ -294,32 +294,7 @@ The Apache Linkis PMC
到此，整个流程才算走完，候选人才正式的成为项目的Committer或者PMC成员。
-## 4 更新项目状态信息
-step1 克隆状态记录的文件
-```shell script
-svn co https://svn.apache.org/repos/asf/public/trunk/content/projects/
-```
-
-step2 修改 linkis.xml 中的new信息
-增加committer信息记录
-```shell script
-<section id="News">
- <title>News</title>
- <ul>
-<!-- <li>YYYY-MM-DD New committer: Fred Hampton</li> -->
- <li>2021-08-02 Project enters incubation.</li>
- <li>2022-02-19 First Apache Linkis release v1.0.3</li>
- <li>2022-02-24 New Committer: Chen Xia</li>
- <li>2022-04-15 Apache Linkis release v1.1.0</li>
-
- </ul>
- </section>
-```
-step3 更新提交后，信息会在这里呈现
-https://incubator.apache.org/projects/linkis.html
-
-
-## 5 操作流程总结
+## 4 操作流程总结
1. 发送携带ICLA附件的邮件
2. 1-2天后收到回复邮件，将在5个工作日内处理
3. 2-5天内收到apache账户创建成功邮件
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/architecture/feature/commons/variable.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/architecture/feature/commons/variable.md
index dd3137da9a..ebcca3464c 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/architecture/feature/commons/variable.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/architecture/feature/commons/variable.md
@@ -4,12 +4,16 @@ sidebar_position: 1
---
## 1. 总述
-### 1.1 需求背景
- 用户希望在写代码时，能够定义一些公共变量然后执行的时候进行替换，比如用户每天都会批量跑同一段sql，需要指定上一天的分区时间，如果基于sql去写会比较复杂如果系统提供一个run_date的变量将会非常方便使用。
-### 1.2 目标
+### 需求
+1.
用户希望Linkis能够提供一些公共变量然后执行的时候进行替换，比如用户每天都会批量跑同一段sql，需要指定上一天的分区时间，如果基于sql去写会比较复杂如果系统提供一个run_date的变量将会非常方便使用。
+2. 用户希望Linkis支持date pattern计算，支持在代码中写&{YYYY-MM-DD}等变量计算时间变量
+3. 用户希望自己定义变量，比如定一个一个float变量，然后在代码中进行使用
+
+### 目标
1. 支持任务代码的变量替换
2. 支持自定义变量，支持用户在脚本和提交给Linkis的任务参数定义自定义变量，支持简单的+，-等计算
3. 预设置系统变量：run_date,run_month，run_today等系统变量
+4. 支持date pattern变量，支持pattern的+，-操作
## 2. 总体设计
在Linkis任务执行过程中自定义变量在Entrance进行，主要通过Entrance在任务提交执行前的拦截器进行拦截替换实现，通过正则表达式获取到任务代码中使用到的变量和定义的变量，并通过任务传入的自定义变量初始值完成代码的替换，变成最终可以执行的代码。
@@ -17,60 +21,37 @@ sidebar_position: 1
### 2.1 技术架构
自定义变量整体架构如下，用于任务提交过来后，会经过变量替换拦截器。首先会解析出所有代码中用到的变量和表达式，然后通过和系统以及用户自定义的变量初始值进行替换，最终将解析后的代码提交给EngineConn执行。所以到底层引擎已经是替换好的代码。
-
-
-## 3.功能介绍
- Linkis支持的变量类型分为自定义变量和系统内置变量，内部变量是Linkis预先定义好的，可以直接进行使用。然后不同的变量类型支持不同的计算格式：String支持+、整数小数支持+-*/,日期支持+-。
-
-### 3.1 内置变量
-目前已支持的内置变量如下：
-
-| 变量名 | 变量类型 | 变量含义 | 变量值举例 |
-| ------ | -------- | -------- | ---------- |
-| run\_date | String | 数据统计时间（支持用户自己设定，默认设置为当前时间的前一天），如今天执行昨天的数据，则为昨天的时间，格式为
yyyyMMdd | 20180129 |
-| run\_date\_std | String | 数据统计时间(标准日期格式)，如今天执行昨天数据，则为昨天的时间，格式为 yyyy-MM-dd |
2018-01-29 |
-| run_today | String | run_date (数据统计时间) 的后一天，格式为 yyyyMMdd | 20211210 |
-| run_today_std | String | run_date (数据统计时间) 的后一天（标准格式），格式为 yyyy-MM-dd |
2021-12-10 |
-| run_mon | String | 数据统计时间所在月，格式为 yyyyMM | 202112 |
-| run_mon_std | String | 数据统计时间所在月（标准格式），格式为 yyyy-MM | 2021-12 |
-| run\_month\_begin | String | 数据统计时间所在月的第一天，格式为 yyyyMMdd | 20180101 |
-| run\_month\_begin\_std | String | 数据统计时间所在月的第一天(标准日期格式)，格式为 yyyy-MM-dd |
2018-01-01 |
-| run_month_now_begin | String | run_today 所在月的第一天，格式为 yyyyMMdd | 20211201 |
-| run_month_now_begin_std | String | run_today 所在月的第一天（标准格式），格式为 yyyy-MM-dd |
2021-12-01 |
-| run\_month\_end | String | 数据统计时间所在月的最后一天，格式为 yyyyMMdd | 20180131 |
-| run\_month\_end\_std | String | 数据统计时间所在月的最后一天(标准日期格式)，格式为 yyyy-MM-dd |
2018-01-31 |
-| run_month_now_end | String | run_today 所在月的最后一天，格式为 yyyyMMdd | 20211231 |
-| run_month_now_end_std | String | run_today 所在月的最后一天（标准日期格式），格式为 yyyy-MM-dd |
2021-12-31 |
-| run_quarter_begin | String | 数据统计时间所在季度的第一天，格式为 yyyyMMdd | 20210401 |
-| run_quarter_end | String | 数据统计时间所在季度的最后一天，格式为 yyyyMMdd | 20210630 |
-| run_half_year_begin | String | 数据统计时间所在半年的第一天，格式为 yyyyMMdd | 20210101 |
-| run_half_year_end | String | 数据统计时间所在半年的最后一天，格式为 yyyyMMdd | 20210630 |
-| run_year_begin | String | 数据统计时间所在年的第一天，格式为 yyyyMMdd | 20210101 |
-| run_year_end | String | 数据统计时间所在年的最后一天，格式为 yyyyMMdd | 20211231 |
-| run_quarter_begin_std | String | 数据统计时间所在季度的第一天（标准格式），格式为 yyyy-MM-dd |
2021-10-01 |
-| run_quarter_end_std | String | 数据统计时间所在季度的最后一天（标准格式），格式为 yyyy-MM-dd |
2021-12-31 |
-| run_half_year_begin_std | String | 数据统计时间所在半年的第一天（标准格式），格式为 yyyy-MM-dd |
2021-07-01 |
-| run_half_year_end_std | String | 数据统计时间所在半年的最后一天（标准格式），格式为 yyyy-MM-dd |
2021-12-31 |
-| run_year_begin_std | String | 数据统计时间所在年的第一天（标准格式），格式为 yyyy-MM-dd |
2021-01-01 |
-| run_year_end_std | String | 数据统计时间所在年的最后一天（标准格式），格式为 yyyy-MM-dd | 2021-12-31
|
-
-具体细节：
+
+
+备注：变量和解析的功能因为比较通用，抽成工具类定义在linkis-commons中：org.apache.linkis.common.utils.VariableUtils
+
+### 2.2 业务架构
+ 此次的特性主要是为了完成变量替换的解析、计算、替换功能，主要涉及到linkis的Entrance模块做代码拦截和Linkis-commons模块定义的变量替换的工具类：
+
+| 组件名| 一级模块 | 二级模块 | 功能点 |
+|---|---|---|---|
+| Linkis | CG | Entrance|拦截任务的代码并调用linkis-common的VariableUtils进行代码替换|
+| Linkis | Linkis-commons | linkis-common|提供变量、解析、计算的工具类VariableUtils|
+
+## 3. 模块设计
+### 3.1核心执行流程
+[输入端] 输入端为代码和代码类型（python/sql/scala/sh）.
+[处理流程] Entrance在接受到任务后会首先进过拦截器，启动变量拦截器会完成变量的解析、替换、计算
+整体时序图如下：
+
+
+
+这里需要说明的是：
+1. 自定义变量和系统变量使用方式为${}，如${run_date}
+2. date pattern变量的使用方式为&{}，如&{yyyy-01-01} 值为2022-01-01。
+
之所以分成两种不同的方式是为了防止自定义变量定义的字符串中包含pattern字符，如定义了一个y=1的自定义变量可能代表不同的含义，会被pattern任务是年变量
+
+### 3.2具体细节：
1、run_date为核心自带日期变量，支持用户自定义日期，如果不指定默认为当前系统时间的前一天。
2、其他衍生内置日期变量定义：其他日期内置变量都是相对run_date计算出来的，一旦run_date变化，其他变量值也会自动跟着变化，其他日期变量不支持设置初始值，只能通过修改run_date进行修改。
3、内置变量支持更加丰富的使用场景：${run_date-1}为run_data的前一天；${run_month_begin-1}为run_month_begin的上个月的第一天，这里的-1表示减一个月。
-
-### 3.2 自定义变量
- 什么是自定义变量？先定义，后使用的用户变量。用户自定义变量暂时支持字符串，整数，浮点数变量的定义，其中字符串支持+法，整数和浮点数支持+-*/方法。用户自定义变量与SparkSQL和HQL本身支持的set变量语法不冲突，但是不允许同名。如何定义和使用自定义变量？如下：
-```
-## 代码中定义，在任务代码前进行指定
-sql类型定义方式：
---@set f=20.1
-python/Shell类型定义如下：
-#@set f=20.1
-注意：只支持一行定义一个变量
-```
-使用都是直接在代码中使用通过 ```{varName表达式},如${f*2}```
+4、Pattern类型变量也是基于run_date进行计算，然后再进行替换和+—
### 3.3 变量作用域
在linkis中自定义变量也有作用域，优先级为脚本中定义的变量大于在任务参数中定义的Variable大于内置的run_date变量。任务参数中定义如下：
@@ -103,6 +84,16 @@ JobSubmitAction.builder
.build
```
+## 4. 接口设计：
+主要工具类为：
+```
+VariableUtils:
+def replace(replaceStr: String): String 替换代码中的变量返回替换后的代码
+def replace(replaceStr: String, variables: util.Map[String, Any]): String
支持传入自定义变量的值，进行替换
+def replace(code: String, runtType: String, variables: util.Map[String,
String]): String 支持传入代码类型，按照不同的类型进行替换解析
+```
+
+
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/user-guide/dynamic-variables.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/user-guide/dynamic-variables.md
index 4cf214497b..e6836fe269 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/user-guide/dynamic-variables.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/user-guide/dynamic-variables.md
@@ -3,14 +3,182 @@ title: 内置时间变量
sidebar_position: 6
---
-## 1.总述
-### 1.1 需求背景
-用户希望在写代码时，对时间的格式要求千变万化,已有的[Linkis自定义变量](https://linkis.apache.org/docs/latest/architecture/commons/variable/)目前还不足以支撑这些需求。另外,已有的时间运算-1有些表示减一个月,有些则是减一天,用户很容易混淆
+## 1. 总述
+### 需求
+1.
用户希望Linkis能够提供一些公共变量然后执行的时候进行替换，比如用户每天都会批量跑同一段sql，需要指定上一天的分区时间，如果基于sql去写会比较复杂如果系统提供一个run_date的变量将会非常方便使用。
+2. 用户希望Linkis支持date pattern计算，支持在代码中写&{YYYY-MM-DD}等变量计算时间变量
+3. 用户希望自己定义变量，比如定一个一个float变量，然后在代码中进行使用
+
+### 目标
+1. 支持任务代码的变量替换
+2. 支持自定义变量，支持用户在脚本和提交给Linkis的任务参数定义自定义变量，支持简单的+，-等计算
+3. 预设置系统变量：run_date,run_month，run_today等系统变量
+4. 支持date pattern变量，支持pattern的+，-操作
+
+具体技术架构可以参考：
+[Linkis自定义变量](https://linkis.apache.org/docs/latest/architecture/commons/variable/)
+
+## 2 自定义变量功能介绍
+ Linkis支持的变量类型分为自定义变量和系统内置变量，内部变量是Linkis预先定义好的，可以直接进行使用。然后不同的变量类型支持不同的计算格式：String支持+、整数小数支持+-*/,日期支持+-。
+
+### 2.1 内置变量
+目前已支持的内置变量如下：
+（下表中的变量值举例以闰年的20240229作为特殊日期）
+
+| 变量名 | 变量类型 | 变量含义 | 变量值举例 |
+| ------ | -------- | -------- | ---------- |
+| run\_date | String | 数据统计时间（支持用户自己设定，默认设置为当前时间的前一天），如今天执行昨天的数据，则为昨天的时间，格式为
yyyyMMdd | 20240229 |
+| run\_date\_std | String | 数据统计时间(标准日期格式)，如今天执行昨天数据，则为昨天的时间，格式为 yyyy-MM-dd |
2024-02-29 |
+| run_today | String | run_date (数据统计时间) 的后一天，格式为 yyyyMMdd | 20240301 |
+| run_today_std | String | run_date (数据统计时间) 的后一天（标准格式），格式为 yyyy-MM-dd |
2024-03-01 |
+| run_mon | String | run_date所在月，格式为 yyyyMM | 202402 |
+| run_mon_std | String | run_date所在月（标准格式），格式为 yyyy-MM | 2024-02 |
+| run\_month\_begin | String | 数据统计时间所在月的第一天，格式为 yyyyMMdd | 20240201 |
+| run\_month\_begin\_std | String | run_date 所在月的第一天(标准日期格式)，格式为 yyyy-MM-dd |
2024-02-01 |
+| run_month_now_begin | String | run_today所在上月的第一天，格式为 yyyyMMdd | 20240201 |
+| run_month_now_begin_std | String | run_today所在上月的第一天（标准格式），格式为 yyyy-MM-dd |
2024-02-01 |
+| run\_month\_end | String | run_date所在月的最后一天，格式为 yyyyMMdd | 20240229 |
+| run\_month\_end\_std | String | run_date 所在月的最后一天(标准日期格式)，格式为 yyyy-MM-dd |
2024-02-29 |
+| run_month_now_end | String | run_today 所在上月的最后一天，格式为 yyyyMMdd | 20240229 |
+| run_month_now_end_std | String | run_today 所在上月的最后一天（标准日期格式），格式为 yyyy-MM-dd
| 2024-02-29 |
+| run_quarter_begin | String | run_date所在季度的第一天，格式为 yyyyMMdd | 20240101 |
+| run_quarter_end | String | run_date 所在季度的最后一天，格式为 yyyyMMdd | 20240331 |
+| run_half_year_begin | String | run_date 所在半年的第一天，格式为 yyyyMMdd | 20240101 |
+| run_half_year_end | String | run_date所在半年的最后一天，格式为 yyyyMMdd | 20240630 |
+| run_year_begin | String | run_date所在年的第一天，格式为 yyyyMMdd | 20240101 |
+| run_year_end | String | run_date所在年的最后一天，格式为 yyyyMMdd | 20241231 |
+| run_quarter_begin_std | String | run_date所在季度的第一天（标准格式），格式为 yyyy-MM-dd |
2024-01-01 |
+| run_quarter_end_std | String | run_date所在季度的最后一天（标准格式），格式为 yyyy-MM-dd |
2024-03-31 |
+| run_half_year_begin_std | String | run_date所在半年的第一天（标准格式），格式为 yyyy-MM-dd |
2024-01-01 |
+| run_half_year_end_std | String | run_date所在半年的最后一天（标准格式），格式为 yyyy-MM-dd |
2024-06-30 |
+| run_year_begin_std | String | run_date所在年的第一天（标准格式），格式为 yyyy-MM-dd |
2024-01-01 |
+| run_year_end_std | String | run_date所在年的最后一天（标准格式），格式为 yyyy-MM-dd |
2024-12-31 |
+| run_tody_h | String | run_today任务运行的时间， yyyyMMddHH | 2024030111 |
+| run_tody_h_std | String | run_today任务运行的时间 yyyy-MM-dd HH | 2024-03-01 11 |
+
+具体细节：
+
+1、run_date为核心自带日期变量，支持用户自定义日期，如果不指定默认为当前系统时间的前一天。
+2、其他衍生内置日期变量定义：其他日期内置变量都是相对run_date计算出来的，一旦run_date变化，其他变量值也会自动跟着变化，其他日期变量不支持设置初始值，只能通过修改run_date进行修改。
+3、内置变量支持更加丰富的使用场景：${run_date-1}为run_data的前一天；${run_month_begin-1}为run_month_begin的上个月的第一天，这里的-1表示减一个月。
+
+以sql为例的例子：
+```
+--@set run_date=20240229
+select
+"${run_date}" as run_date,
+"${run_date_std}" as run_date_std,
+"${run_today}" as run_today,
+"${run_today_std}" as run_today_std,
+"${run_mon}" as run_mon,
+"${run_mon_std}" as run_mon_std,
+"${run_month_begin}" as run_month_begin,
+"${run_month_begin_std}" as run_month_begin_std,
+"${run_month_now_begin}" as run_month_now_begin,
+"${run_month_now_begin_std}" as run_month_now_begin_std,
+"${run_month_end}" as run_month_end,
+"${run_month_end_std}" as run_month_end_std,
+"${run_month_now_end}" as run_month_now_end,
+"${run_month_now_end_std}" as run_month_now_end_std,
+"${run_quarter_begin}" as run_quarter_begin,
+"${run_quarter_end}" as run_quarter_end,
+"${run_half_year_begin}" as run_half_year_begin,
+"${run_half_year_end}" as run_half_year_end,
+"${run_year_begin}" as run_year_begin,
+"${run_year_end}" as run_year_end,
+"${run_quarter_begin_std}" as run_quarter_begin_std,
+"${run_quarter_end_std}" as run_quarter_end_std,
+"${run_half_year_begin_std}" as run_half_year_begin_std,
+"${run_half_year_end_std}" as run_half_year_end_std,
+"${run_year_begin_std}" as run_year_begin_std,
+"${run_year_end_std}" as run_year_end_std,
+"${run_today_h}" as run_tody_h,
+"${run_today_h_std}" as run_tody_h_std
+
+编译替换后的内容：
+select
+"20240229" as run_date,
+"2024-02-29" as run_date_std,
+"20240301" as run_today,
+"2024-03-01" as run_today_std,
+"202402" as run_mon,
+"2024-02" as run_mon_std,
+"20240201" as run_month_begin,
+"2024-02-01" as run_month_begin_std,
+"20240201" as run_month_now_begin,
+"2024-02-01" as run_month_now_begin_std,
+"20240229" as run_month_end,
+"2024-02-29" as run_month_end_std,
+"20240229" as run_month_now_end,
+"2024-02-29" as run_month_now_end_std,
+"20240101" as run_quarter_begin,
+"20240331" as run_quarter_end,
+"20240101" as run_half_year_begin,
+"20240630" as run_half_year_end,
+"20240101" as run_year_begin,
+"20241231" as run_year_end,
+"2024-01-01" as run_quarter_begin_std,
+"2024-03-31" as run_quarter_end_std,
+"2024-01-01" as run_half_year_begin_std,
+"2024-06-30" as run_half_year_end_std,
+"2024-01-01" as run_year_begin_std,
+"2024-12-31" as run_year_end_std,
+"2024030111" as run_tody_h,
+"2024-03-01 11" as run_tody_h_std
+
+```
+
+### 2.2 自定义变量
+ 什么是自定义变量？先定义，后使用的用户变量。用户自定义变量暂时支持字符串，整数，浮点数变量的定义，其中字符串支持+法，整数和浮点数支持+-*/方法。用户自定义变量与SparkSQL和HQL本身支持的set变量语法不冲突，但是不允许同名。如何定义和使用自定义变量？如下：
+```
+## 代码中定义，在任务代码前进行指定
+sql类型定义方式：
+--@set f=20.1
+python/Shell类型定义如下：
+#@set f=20.1
+注意：只支持一行定义一个变量
+```
+使用都是直接在代码中使用通过 ```{varName表达式},如${f*2}```
+
+### 2.3 变量作用域
+在linkis中自定义变量也有作用域，优先级为脚本中定义的变量大于在任务参数中定义的Variable大于内置的run_date变量。任务参数中定义如下：
+```
+## restful
+{
+ "executionContent": {"code": "select \"${f-1}\";", "runType": "sql"},
+ "params": {
+ "variable": {f: "20.1"},
+ "configuration": {
+ "runtime": {
+
"linkis.openlookeng.url":"http://127.0.0.1:9090";
+ }
+ }
+ },
+ "source": {"scriptPath": "file:///mnt/bdp/hadoop/1.sql"},
+ "labels": {
+ "engineType": "spark-2.4.3",
+ "userCreator": "hadoop-IDE"
+ }
+}
+## java SDK
+JobSubmitAction.builder
+ .addExecuteCode(code)
+ .setStartupParams(startupMap)
+ .setUser(user) //submit user
+ .addExecuteUser(user) //execute user
+ .setLabels(labels)
+ .setVariableMap(varMap) //setVar
+ .build
+```
+
+## 3. Date Pattern变量介绍
+
+### 3.1 功能
-### 1.2 目标
-* 其他日期内置变量都是相对run_date计算出来
* 支持Pattern格式时间且用户可以自行随意指定
+* Pattern日期变量都是相对run_date计算出来的
* 支持±y/±M/±d/±H等等
+* 其中+-是先对linkis内置参数run_date进行运算,然后将%之前的pattern字段进行替换,非Pattern字符不支持运算替换。
Pattern格式对照表:
@@ -26,7 +194,7 @@ D |Day in year |Number| 189
d |Day in month |Number |10
F |Day of week in month| Number |2
E |Day name in week |Text |Tuesday; Tue
-e |Day number of week (1 = Monday, …, 7 = Sunday) |Number |1
+u |Day number of week (1 = Monday, …, 7 = Sunday) |Number |1
a |Am/pm marker |Text |PM
H |Hour in day (0-23) |Number| 0
k |Hour in day (1-24)| Number |24
@@ -39,14 +207,8 @@ z |Time zone |General time zone |Pacific
Standard Time; PST; GMT-08:00
Z |Time zone |RFC 822 time zone |-0800
X |Time zone |ISO 8601 time zone |-08; -0800; -08:00
-## 2. 总体设计
-总体设计和技术架构参照[Linkis自定义变量](https://linkis.apache.org/docs/latest/architecture/commons/variable/)
-## 3. 功能介绍
-* Linkis支持的变量类型分为自定义变量(不做赘述)和系统内置变量，自定变量日期支持+-。
-* 其中+-是先对linkis内置参数run_date进行运算,然后将%之前的pattern字段进行替换,非Pattern字符不支持运算替换。
-
-### 3.1 内置变量举例
+### 3.2 Date Pattern变量举例
可以根据自己的喜好/业务实际情况定义需要动态渲染的参数
variable | result
@@ -60,7 +222,7 @@ variable | result
&{yyyyMM01%-1M} | 20210301
&{HH%-1H} | 14
-### 3.2 自定义变量使用实例
+### 3.3 Date Pattern变量使用实例
* 例子1: sql
```sql
@@ -177,4 +339,4 @@ print(&{yyyyMMdd%-1d})
渲染后
```
20220705
-```
+```
\ No newline at end of file
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.3.0/user-guide/dynamic-variables.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.3.0/user-guide/dynamic-variables.md
index 347d407956..e6836fe269 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.3.0/user-guide/dynamic-variables.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.3.0/user-guide/dynamic-variables.md
@@ -1,16 +1,184 @@
---
title: 内置时间变量
-sidebar_position: 7
+sidebar_position: 6
---
-## 1.总述
-### 需求背景
-用户希望在写代码时，对时间的格式要求千变万化,已有的[Linkis自定义变量](https://linkis.apache.org/docs/latest/architecture/commons/variable/)目前还不足以支撑这些需求。另外,已有的时间运算-1有些表示减一个月,有些则是减一天,用户很容易混淆
+## 1. 总述
+### 需求
+1.
用户希望Linkis能够提供一些公共变量然后执行的时候进行替换，比如用户每天都会批量跑同一段sql，需要指定上一天的分区时间，如果基于sql去写会比较复杂如果系统提供一个run_date的变量将会非常方便使用。
+2. 用户希望Linkis支持date pattern计算，支持在代码中写&{YYYY-MM-DD}等变量计算时间变量
+3. 用户希望自己定义变量，比如定一个一个float变量，然后在代码中进行使用
### 目标
-* 其他日期内置变量都是相对run_date计算出来
+1. 支持任务代码的变量替换
+2. 支持自定义变量，支持用户在脚本和提交给Linkis的任务参数定义自定义变量，支持简单的+，-等计算
+3. 预设置系统变量：run_date,run_month，run_today等系统变量
+4. 支持date pattern变量，支持pattern的+，-操作
+
+具体技术架构可以参考：
+[Linkis自定义变量](https://linkis.apache.org/docs/latest/architecture/commons/variable/)
+
+## 2 自定义变量功能介绍
+ Linkis支持的变量类型分为自定义变量和系统内置变量，内部变量是Linkis预先定义好的，可以直接进行使用。然后不同的变量类型支持不同的计算格式：String支持+、整数小数支持+-*/,日期支持+-。
+
+### 2.1 内置变量
+目前已支持的内置变量如下：
+（下表中的变量值举例以闰年的20240229作为特殊日期）
+
+| 变量名 | 变量类型 | 变量含义 | 变量值举例 |
+| ------ | -------- | -------- | ---------- |
+| run\_date | String | 数据统计时间（支持用户自己设定，默认设置为当前时间的前一天），如今天执行昨天的数据，则为昨天的时间，格式为
yyyyMMdd | 20240229 |
+| run\_date\_std | String | 数据统计时间(标准日期格式)，如今天执行昨天数据，则为昨天的时间，格式为 yyyy-MM-dd |
2024-02-29 |
+| run_today | String | run_date (数据统计时间) 的后一天，格式为 yyyyMMdd | 20240301 |
+| run_today_std | String | run_date (数据统计时间) 的后一天（标准格式），格式为 yyyy-MM-dd |
2024-03-01 |
+| run_mon | String | run_date所在月，格式为 yyyyMM | 202402 |
+| run_mon_std | String | run_date所在月（标准格式），格式为 yyyy-MM | 2024-02 |
+| run\_month\_begin | String | 数据统计时间所在月的第一天，格式为 yyyyMMdd | 20240201 |
+| run\_month\_begin\_std | String | run_date 所在月的第一天(标准日期格式)，格式为 yyyy-MM-dd |
2024-02-01 |
+| run_month_now_begin | String | run_today所在上月的第一天，格式为 yyyyMMdd | 20240201 |
+| run_month_now_begin_std | String | run_today所在上月的第一天（标准格式），格式为 yyyy-MM-dd |
2024-02-01 |
+| run\_month\_end | String | run_date所在月的最后一天，格式为 yyyyMMdd | 20240229 |
+| run\_month\_end\_std | String | run_date 所在月的最后一天(标准日期格式)，格式为 yyyy-MM-dd |
2024-02-29 |
+| run_month_now_end | String | run_today 所在上月的最后一天，格式为 yyyyMMdd | 20240229 |
+| run_month_now_end_std | String | run_today 所在上月的最后一天（标准日期格式），格式为 yyyy-MM-dd
| 2024-02-29 |
+| run_quarter_begin | String | run_date所在季度的第一天，格式为 yyyyMMdd | 20240101 |
+| run_quarter_end | String | run_date 所在季度的最后一天，格式为 yyyyMMdd | 20240331 |
+| run_half_year_begin | String | run_date 所在半年的第一天，格式为 yyyyMMdd | 20240101 |
+| run_half_year_end | String | run_date所在半年的最后一天，格式为 yyyyMMdd | 20240630 |
+| run_year_begin | String | run_date所在年的第一天，格式为 yyyyMMdd | 20240101 |
+| run_year_end | String | run_date所在年的最后一天，格式为 yyyyMMdd | 20241231 |
+| run_quarter_begin_std | String | run_date所在季度的第一天（标准格式），格式为 yyyy-MM-dd |
2024-01-01 |
+| run_quarter_end_std | String | run_date所在季度的最后一天（标准格式），格式为 yyyy-MM-dd |
2024-03-31 |
+| run_half_year_begin_std | String | run_date所在半年的第一天（标准格式），格式为 yyyy-MM-dd |
2024-01-01 |
+| run_half_year_end_std | String | run_date所在半年的最后一天（标准格式），格式为 yyyy-MM-dd |
2024-06-30 |
+| run_year_begin_std | String | run_date所在年的第一天（标准格式），格式为 yyyy-MM-dd |
2024-01-01 |
+| run_year_end_std | String | run_date所在年的最后一天（标准格式），格式为 yyyy-MM-dd |
2024-12-31 |
+| run_tody_h | String | run_today任务运行的时间， yyyyMMddHH | 2024030111 |
+| run_tody_h_std | String | run_today任务运行的时间 yyyy-MM-dd HH | 2024-03-01 11 |
+
+具体细节：
+
+1、run_date为核心自带日期变量，支持用户自定义日期，如果不指定默认为当前系统时间的前一天。
+2、其他衍生内置日期变量定义：其他日期内置变量都是相对run_date计算出来的，一旦run_date变化，其他变量值也会自动跟着变化，其他日期变量不支持设置初始值，只能通过修改run_date进行修改。
+3、内置变量支持更加丰富的使用场景：${run_date-1}为run_data的前一天；${run_month_begin-1}为run_month_begin的上个月的第一天，这里的-1表示减一个月。
+
+以sql为例的例子：
+```
+--@set run_date=20240229
+select
+"${run_date}" as run_date,
+"${run_date_std}" as run_date_std,
+"${run_today}" as run_today,
+"${run_today_std}" as run_today_std,
+"${run_mon}" as run_mon,
+"${run_mon_std}" as run_mon_std,
+"${run_month_begin}" as run_month_begin,
+"${run_month_begin_std}" as run_month_begin_std,
+"${run_month_now_begin}" as run_month_now_begin,
+"${run_month_now_begin_std}" as run_month_now_begin_std,
+"${run_month_end}" as run_month_end,
+"${run_month_end_std}" as run_month_end_std,
+"${run_month_now_end}" as run_month_now_end,
+"${run_month_now_end_std}" as run_month_now_end_std,
+"${run_quarter_begin}" as run_quarter_begin,
+"${run_quarter_end}" as run_quarter_end,
+"${run_half_year_begin}" as run_half_year_begin,
+"${run_half_year_end}" as run_half_year_end,
+"${run_year_begin}" as run_year_begin,
+"${run_year_end}" as run_year_end,
+"${run_quarter_begin_std}" as run_quarter_begin_std,
+"${run_quarter_end_std}" as run_quarter_end_std,
+"${run_half_year_begin_std}" as run_half_year_begin_std,
+"${run_half_year_end_std}" as run_half_year_end_std,
+"${run_year_begin_std}" as run_year_begin_std,
+"${run_year_end_std}" as run_year_end_std,
+"${run_today_h}" as run_tody_h,
+"${run_today_h_std}" as run_tody_h_std
+
+编译替换后的内容：
+select
+"20240229" as run_date,
+"2024-02-29" as run_date_std,
+"20240301" as run_today,
+"2024-03-01" as run_today_std,
+"202402" as run_mon,
+"2024-02" as run_mon_std,
+"20240201" as run_month_begin,
+"2024-02-01" as run_month_begin_std,
+"20240201" as run_month_now_begin,
+"2024-02-01" as run_month_now_begin_std,
+"20240229" as run_month_end,
+"2024-02-29" as run_month_end_std,
+"20240229" as run_month_now_end,
+"2024-02-29" as run_month_now_end_std,
+"20240101" as run_quarter_begin,
+"20240331" as run_quarter_end,
+"20240101" as run_half_year_begin,
+"20240630" as run_half_year_end,
+"20240101" as run_year_begin,
+"20241231" as run_year_end,
+"2024-01-01" as run_quarter_begin_std,
+"2024-03-31" as run_quarter_end_std,
+"2024-01-01" as run_half_year_begin_std,
+"2024-06-30" as run_half_year_end_std,
+"2024-01-01" as run_year_begin_std,
+"2024-12-31" as run_year_end_std,
+"2024030111" as run_tody_h,
+"2024-03-01 11" as run_tody_h_std
+
+```
+
+### 2.2 自定义变量
+ 什么是自定义变量？先定义，后使用的用户变量。用户自定义变量暂时支持字符串，整数，浮点数变量的定义，其中字符串支持+法，整数和浮点数支持+-*/方法。用户自定义变量与SparkSQL和HQL本身支持的set变量语法不冲突，但是不允许同名。如何定义和使用自定义变量？如下：
+```
+## 代码中定义，在任务代码前进行指定
+sql类型定义方式：
+--@set f=20.1
+python/Shell类型定义如下：
+#@set f=20.1
+注意：只支持一行定义一个变量
+```
+使用都是直接在代码中使用通过 ```{varName表达式},如${f*2}```
+
+### 2.3 变量作用域
+在linkis中自定义变量也有作用域，优先级为脚本中定义的变量大于在任务参数中定义的Variable大于内置的run_date变量。任务参数中定义如下：
+```
+## restful
+{
+ "executionContent": {"code": "select \"${f-1}\";", "runType": "sql"},
+ "params": {
+ "variable": {f: "20.1"},
+ "configuration": {
+ "runtime": {
+
"linkis.openlookeng.url":"http://127.0.0.1:9090";
+ }
+ }
+ },
+ "source": {"scriptPath": "file:///mnt/bdp/hadoop/1.sql"},
+ "labels": {
+ "engineType": "spark-2.4.3",
+ "userCreator": "hadoop-IDE"
+ }
+}
+## java SDK
+JobSubmitAction.builder
+ .addExecuteCode(code)
+ .setStartupParams(startupMap)
+ .setUser(user) //submit user
+ .addExecuteUser(user) //execute user
+ .setLabels(labels)
+ .setVariableMap(varMap) //setVar
+ .build
+```
+
+## 3. Date Pattern变量介绍
+
+### 3.1 功能
+
* 支持Pattern格式时间且用户可以自行随意指定
+* Pattern日期变量都是相对run_date计算出来的
* 支持±y/±M/±d/±H等等
+* 其中+-是先对linkis内置参数run_date进行运算,然后将%之前的pattern字段进行替换,非Pattern字符不支持运算替换。
Pattern格式对照表:
@@ -26,7 +194,7 @@ D |Day in year |Number| 189
d |Day in month |Number |10
F |Day of week in month| Number |2
E |Day name in week |Text |Tuesday; Tue
-e |Day number of week (1 = Monday, …, 7 = Sunday) |Number |1
+u |Day number of week (1 = Monday, …, 7 = Sunday) |Number |1
a |Am/pm marker |Text |PM
H |Hour in day (0-23) |Number| 0
k |Hour in day (1-24)| Number |24
@@ -39,14 +207,8 @@ z |Time zone |General time zone |Pacific
Standard Time; PST; GMT-08:00
Z |Time zone |RFC 822 time zone |-0800
X |Time zone |ISO 8601 time zone |-08; -0800; -08:00
-## 2. 总体设计
-总体设计和技术架构参照[Linkis自定义变量](https://linkis.apache.org/docs/latest/architecture/commons/variable/)
-## 3. 功能介绍
-* Linkis支持的变量类型分为自定义变量(不做赘述)和系统内置变量，自定变量日期支持+-。
-* 其中+-是先对linkis内置参数run_date进行运算,然后将%之前的pattern字段进行替换,非Pattern字符不支持运算替换。
-
-### 3.1 内置变量举例
+### 3.2 Date Pattern变量举例
可以根据自己的喜好/业务实际情况定义需要动态渲染的参数
variable | result
@@ -60,7 +222,7 @@ variable | result
&{yyyyMM01%-1M} | 20210301
&{HH%-1H} | 14
-### 3.2 自定义变量使用实例
+### 3.3 Date Pattern变量使用实例
* 例子1: sql
```sql
@@ -177,4 +339,4 @@ print(&{yyyyMMdd%-1d})
渲染后
```
20220705
-```
+```
\ No newline at end of file
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.3.1/user-guide/dynamic-variables.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.3.1/user-guide/dynamic-variables.md
index 4cf214497b..e6836fe269 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.3.1/user-guide/dynamic-variables.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.3.1/user-guide/dynamic-variables.md
@@ -3,14 +3,182 @@ title: 内置时间变量
sidebar_position: 6
---
-## 1.总述
-### 1.1 需求背景
-用户希望在写代码时，对时间的格式要求千变万化,已有的[Linkis自定义变量](https://linkis.apache.org/docs/latest/architecture/commons/variable/)目前还不足以支撑这些需求。另外,已有的时间运算-1有些表示减一个月,有些则是减一天,用户很容易混淆
+## 1. 总述
+### 需求
+1.
用户希望Linkis能够提供一些公共变量然后执行的时候进行替换，比如用户每天都会批量跑同一段sql，需要指定上一天的分区时间，如果基于sql去写会比较复杂如果系统提供一个run_date的变量将会非常方便使用。
+2. 用户希望Linkis支持date pattern计算，支持在代码中写&{YYYY-MM-DD}等变量计算时间变量
+3. 用户希望自己定义变量，比如定一个一个float变量，然后在代码中进行使用
+
+### 目标
+1. 支持任务代码的变量替换
+2. 支持自定义变量，支持用户在脚本和提交给Linkis的任务参数定义自定义变量，支持简单的+，-等计算
+3. 预设置系统变量：run_date,run_month，run_today等系统变量
+4. 支持date pattern变量，支持pattern的+，-操作
+
+具体技术架构可以参考：
+[Linkis自定义变量](https://linkis.apache.org/docs/latest/architecture/commons/variable/)
+
+## 2 自定义变量功能介绍
+ Linkis支持的变量类型分为自定义变量和系统内置变量，内部变量是Linkis预先定义好的，可以直接进行使用。然后不同的变量类型支持不同的计算格式：String支持+、整数小数支持+-*/,日期支持+-。
+
+### 2.1 内置变量
+目前已支持的内置变量如下：
+（下表中的变量值举例以闰年的20240229作为特殊日期）
+
+| 变量名 | 变量类型 | 变量含义 | 变量值举例 |
+| ------ | -------- | -------- | ---------- |
+| run\_date | String | 数据统计时间（支持用户自己设定，默认设置为当前时间的前一天），如今天执行昨天的数据，则为昨天的时间，格式为
yyyyMMdd | 20240229 |
+| run\_date\_std | String | 数据统计时间(标准日期格式)，如今天执行昨天数据，则为昨天的时间，格式为 yyyy-MM-dd |
2024-02-29 |
+| run_today | String | run_date (数据统计时间) 的后一天，格式为 yyyyMMdd | 20240301 |
+| run_today_std | String | run_date (数据统计时间) 的后一天（标准格式），格式为 yyyy-MM-dd |
2024-03-01 |
+| run_mon | String | run_date所在月，格式为 yyyyMM | 202402 |
+| run_mon_std | String | run_date所在月（标准格式），格式为 yyyy-MM | 2024-02 |
+| run\_month\_begin | String | 数据统计时间所在月的第一天，格式为 yyyyMMdd | 20240201 |
+| run\_month\_begin\_std | String | run_date 所在月的第一天(标准日期格式)，格式为 yyyy-MM-dd |
2024-02-01 |
+| run_month_now_begin | String | run_today所在上月的第一天，格式为 yyyyMMdd | 20240201 |
+| run_month_now_begin_std | String | run_today所在上月的第一天（标准格式），格式为 yyyy-MM-dd |
2024-02-01 |
+| run\_month\_end | String | run_date所在月的最后一天，格式为 yyyyMMdd | 20240229 |
+| run\_month\_end\_std | String | run_date 所在月的最后一天(标准日期格式)，格式为 yyyy-MM-dd |
2024-02-29 |
+| run_month_now_end | String | run_today 所在上月的最后一天，格式为 yyyyMMdd | 20240229 |
+| run_month_now_end_std | String | run_today 所在上月的最后一天（标准日期格式），格式为 yyyy-MM-dd
| 2024-02-29 |
+| run_quarter_begin | String | run_date所在季度的第一天，格式为 yyyyMMdd | 20240101 |
+| run_quarter_end | String | run_date 所在季度的最后一天，格式为 yyyyMMdd | 20240331 |
+| run_half_year_begin | String | run_date 所在半年的第一天，格式为 yyyyMMdd | 20240101 |
+| run_half_year_end | String | run_date所在半年的最后一天，格式为 yyyyMMdd | 20240630 |
+| run_year_begin | String | run_date所在年的第一天，格式为 yyyyMMdd | 20240101 |
+| run_year_end | String | run_date所在年的最后一天，格式为 yyyyMMdd | 20241231 |
+| run_quarter_begin_std | String | run_date所在季度的第一天（标准格式），格式为 yyyy-MM-dd |
2024-01-01 |
+| run_quarter_end_std | String | run_date所在季度的最后一天（标准格式），格式为 yyyy-MM-dd |
2024-03-31 |
+| run_half_year_begin_std | String | run_date所在半年的第一天（标准格式），格式为 yyyy-MM-dd |
2024-01-01 |
+| run_half_year_end_std | String | run_date所在半年的最后一天（标准格式），格式为 yyyy-MM-dd |
2024-06-30 |
+| run_year_begin_std | String | run_date所在年的第一天（标准格式），格式为 yyyy-MM-dd |
2024-01-01 |
+| run_year_end_std | String | run_date所在年的最后一天（标准格式），格式为 yyyy-MM-dd |
2024-12-31 |
+| run_tody_h | String | run_today任务运行的时间， yyyyMMddHH | 2024030111 |
+| run_tody_h_std | String | run_today任务运行的时间 yyyy-MM-dd HH | 2024-03-01 11 |
+
+具体细节：
+
+1、run_date为核心自带日期变量，支持用户自定义日期，如果不指定默认为当前系统时间的前一天。
+2、其他衍生内置日期变量定义：其他日期内置变量都是相对run_date计算出来的，一旦run_date变化，其他变量值也会自动跟着变化，其他日期变量不支持设置初始值，只能通过修改run_date进行修改。
+3、内置变量支持更加丰富的使用场景：${run_date-1}为run_data的前一天；${run_month_begin-1}为run_month_begin的上个月的第一天，这里的-1表示减一个月。
+
+以sql为例的例子：
+```
+--@set run_date=20240229
+select
+"${run_date}" as run_date,
+"${run_date_std}" as run_date_std,
+"${run_today}" as run_today,
+"${run_today_std}" as run_today_std,
+"${run_mon}" as run_mon,
+"${run_mon_std}" as run_mon_std,
+"${run_month_begin}" as run_month_begin,
+"${run_month_begin_std}" as run_month_begin_std,
+"${run_month_now_begin}" as run_month_now_begin,
+"${run_month_now_begin_std}" as run_month_now_begin_std,
+"${run_month_end}" as run_month_end,
+"${run_month_end_std}" as run_month_end_std,
+"${run_month_now_end}" as run_month_now_end,
+"${run_month_now_end_std}" as run_month_now_end_std,
+"${run_quarter_begin}" as run_quarter_begin,
+"${run_quarter_end}" as run_quarter_end,
+"${run_half_year_begin}" as run_half_year_begin,
+"${run_half_year_end}" as run_half_year_end,
+"${run_year_begin}" as run_year_begin,
+"${run_year_end}" as run_year_end,
+"${run_quarter_begin_std}" as run_quarter_begin_std,
+"${run_quarter_end_std}" as run_quarter_end_std,
+"${run_half_year_begin_std}" as run_half_year_begin_std,
+"${run_half_year_end_std}" as run_half_year_end_std,
+"${run_year_begin_std}" as run_year_begin_std,
+"${run_year_end_std}" as run_year_end_std,
+"${run_today_h}" as run_tody_h,
+"${run_today_h_std}" as run_tody_h_std
+
+编译替换后的内容：
+select
+"20240229" as run_date,
+"2024-02-29" as run_date_std,
+"20240301" as run_today,
+"2024-03-01" as run_today_std,
+"202402" as run_mon,
+"2024-02" as run_mon_std,
+"20240201" as run_month_begin,
+"2024-02-01" as run_month_begin_std,
+"20240201" as run_month_now_begin,
+"2024-02-01" as run_month_now_begin_std,
+"20240229" as run_month_end,
+"2024-02-29" as run_month_end_std,
+"20240229" as run_month_now_end,
+"2024-02-29" as run_month_now_end_std,
+"20240101" as run_quarter_begin,
+"20240331" as run_quarter_end,
+"20240101" as run_half_year_begin,
+"20240630" as run_half_year_end,
+"20240101" as run_year_begin,
+"20241231" as run_year_end,
+"2024-01-01" as run_quarter_begin_std,
+"2024-03-31" as run_quarter_end_std,
+"2024-01-01" as run_half_year_begin_std,
+"2024-06-30" as run_half_year_end_std,
+"2024-01-01" as run_year_begin_std,
+"2024-12-31" as run_year_end_std,
+"2024030111" as run_tody_h,
+"2024-03-01 11" as run_tody_h_std
+
+```
+
+### 2.2 自定义变量
+ 什么是自定义变量？先定义，后使用的用户变量。用户自定义变量暂时支持字符串，整数，浮点数变量的定义，其中字符串支持+法，整数和浮点数支持+-*/方法。用户自定义变量与SparkSQL和HQL本身支持的set变量语法不冲突，但是不允许同名。如何定义和使用自定义变量？如下：
+```
+## 代码中定义，在任务代码前进行指定
+sql类型定义方式：
+--@set f=20.1
+python/Shell类型定义如下：
+#@set f=20.1
+注意：只支持一行定义一个变量
+```
+使用都是直接在代码中使用通过 ```{varName表达式},如${f*2}```
+
+### 2.3 变量作用域
+在linkis中自定义变量也有作用域，优先级为脚本中定义的变量大于在任务参数中定义的Variable大于内置的run_date变量。任务参数中定义如下：
+```
+## restful
+{
+ "executionContent": {"code": "select \"${f-1}\";", "runType": "sql"},
+ "params": {
+ "variable": {f: "20.1"},
+ "configuration": {
+ "runtime": {
+
"linkis.openlookeng.url":"http://127.0.0.1:9090";
+ }
+ }
+ },
+ "source": {"scriptPath": "file:///mnt/bdp/hadoop/1.sql"},
+ "labels": {
+ "engineType": "spark-2.4.3",
+ "userCreator": "hadoop-IDE"
+ }
+}
+## java SDK
+JobSubmitAction.builder
+ .addExecuteCode(code)
+ .setStartupParams(startupMap)
+ .setUser(user) //submit user
+ .addExecuteUser(user) //execute user
+ .setLabels(labels)
+ .setVariableMap(varMap) //setVar
+ .build
+```
+
+## 3. Date Pattern变量介绍
+
+### 3.1 功能
-### 1.2 目标
-* 其他日期内置变量都是相对run_date计算出来
* 支持Pattern格式时间且用户可以自行随意指定
+* Pattern日期变量都是相对run_date计算出来的
* 支持±y/±M/±d/±H等等
+* 其中+-是先对linkis内置参数run_date进行运算,然后将%之前的pattern字段进行替换,非Pattern字符不支持运算替换。
Pattern格式对照表:
@@ -26,7 +194,7 @@ D |Day in year |Number| 189
d |Day in month |Number |10
F |Day of week in month| Number |2
E |Day name in week |Text |Tuesday; Tue
-e |Day number of week (1 = Monday, …, 7 = Sunday) |Number |1
+u |Day number of week (1 = Monday, …, 7 = Sunday) |Number |1
a |Am/pm marker |Text |PM
H |Hour in day (0-23) |Number| 0
k |Hour in day (1-24)| Number |24
@@ -39,14 +207,8 @@ z |Time zone |General time zone |Pacific
Standard Time; PST; GMT-08:00
Z |Time zone |RFC 822 time zone |-0800
X |Time zone |ISO 8601 time zone |-08; -0800; -08:00
-## 2. 总体设计
-总体设计和技术架构参照[Linkis自定义变量](https://linkis.apache.org/docs/latest/architecture/commons/variable/)
-## 3. 功能介绍
-* Linkis支持的变量类型分为自定义变量(不做赘述)和系统内置变量，自定变量日期支持+-。
-* 其中+-是先对linkis内置参数run_date进行运算,然后将%之前的pattern字段进行替换,非Pattern字符不支持运算替换。
-
-### 3.1 内置变量举例
+### 3.2 Date Pattern变量举例
可以根据自己的喜好/业务实际情况定义需要动态渲染的参数
variable | result
@@ -60,7 +222,7 @@ variable | result
&{yyyyMM01%-1M} | 20210301
&{HH%-1H} | 14
-### 3.2 自定义变量使用实例
+### 3.3 Date Pattern变量使用实例
* 例子1: sql
```sql
@@ -177,4 +339,4 @@ print(&{yyyyMMdd%-1d})
渲染后
```
20220705
-```
+```
\ No newline at end of file
diff --git a/static/Images/Architecture/Commons/var_time.png
b/static/Images/Architecture/Commons/var_time.png
new file mode 100644
index 0000000000..6072775ac2
Binary files /dev/null and b/static/Images/Architecture/Commons/var_time.png
differ
diff --git a/versioned_docs/version-1.3.0/user-guide/dynamic-variables.md
b/versioned_docs/version-1.3.0/user-guide/dynamic-variables.md
index deaa818ee4..77b2d8b5fb 100644
--- a/versioned_docs/version-1.3.0/user-guide/dynamic-variables.md
+++ b/versioned_docs/version-1.3.0/user-guide/dynamic-variables.md
@@ -1,16 +1,184 @@
---
-title: Built-in Time Variable
-sidebar_position: 7
+title: built-in time variable
+sidebar_position: 6
---
-## 1. General
-### Requirements Background
-Users hope that when writing code, the time format requirements are
ever-changing, and the existing [Linkis custom
variables](https://linkis.apache.org/docs/latest/architecture/commons/variable/)
is currently not enough to support these requirements. In addition, some of
the existing time operation -1 means minus one month, and some minus one day,
which is easy for users to confuse
+## 1. Overview
+### need
+1. The user hopes that Linkis can provide some public variables and then
replace them during execution. For example, the user runs the same SQL in
batches every day, and needs to specify the partition time of the previous day.
Writing based on SQL will be more complicated if the system provides a run_date
variable It will be very convenient to use.
+2. The user hopes that Linkis supports date pattern calculation, supports
writing variables such as &{YYYY-MM-DD} in the code to calculate time variables
+3. The user wants to define variables by himself, such as setting a float
variable, and then use it in the code
### Target
-* Other date built-in variables are calculated relative to run_date
-* Support Pattern format time and users can specify at will
+1. Support variable replacement of task code
+2. Support custom variables, support users to define custom variables in
scripts and task parameters submitted to Linkis, support simple +, - and other
calculations
+3. Preset system variables: run_date, run_month, run_today and other system
variables
+4. Support date pattern variable, support +, - operation of pattern
+
+The specific technical architecture can refer to:
+[Linkis Custom
Variables](https://linkis.apache.org/docs/latest/architecture/commons/variable/)
+
+## 2 Custom variable function introduction
+ The variable types supported by Linkis are
divided into custom variables and system built-in variables. The internal
variables are pre-defined by Linkis and can be used directly. Then different
variable types support different calculation formats: String supports +,
integers and decimals support +-*/, and dates support +-.
+
+### 2.1 Built-in variables
+Currently supported built-in variables are as follows:
+(The variable values in the following table take 20240229 in leap year as a
special date as an example)
+
+| variable name | variable type | variable meaning | variable value example |
+| ------ | -------- | -------- | ---------- |
+| run\_date | String | Data statistics time (users can set it themselves, the
default setting is the day before the current time), if you execute yesterday’s
data today, it will be yesterday’s time, the format is yyyyMMdd | 20240229 |
+| run\_date\_std | String | Data statistics time (standard date format), if
yesterday’s data is executed today, it is yesterday’s time, the format is
yyyy-MM-dd | 2024-02-29 |
+| run_today | String | The day after run_date (data statistics time), the
format is yyyyMMdd | 20240301 |
+| run_today_std | String | The day after run_date (data statistics time)
(standard format), the format is yyyy-MM-dd | 2024-03-01 |
+| run_mon | String | The month of run_date, the format is yyyyMM | 202402 |
+| run_mon_std | String | The month of run_date (standard format), the format
is yyyy-MM | 2024-02 |
+| run\_month\_begin | String | The first day of the month where the data
statistics time is located, the format is yyyyMMdd | 20240201 |
+| run\_month\_begin\_std | String | The first day of the month where run_date
is located (standard date format), the format is yyyy-MM-dd | 2024-02-01 |
+| run_month_now_begin | String | the first day of the previous month where
run_today is located, the format is yyyyMMdd | 20240201 |
+| run_month_now_begin_std | String | The first day of the previous month where
run_today is located (standard format), the format is yyyy-MM-dd | 2024-02-01 |
+| run\_month\_end | String | The last day of the month where run_date belongs,
in the format of yyyyMMdd | 20240229 |
+| run\_month\_end\_std | String | The last day of the month of run_date
(standard date format), the format is yyyy-MM-dd | 2024-02-29 |
+| run_month_now_end | String | the last day of the previous month where
run_today is located, the format is yyyyMMdd | 20240229 |
+| run_month_now_end_std | String | The last day of the previous month where
run_today is located (standard date format), the format is yyyy-MM-dd |
2024-02-29 |
+| run_quarter_begin | String | The first day of the quarter where run_date
belongs, the format is yyyyMMdd | 20240101 |
+| run_quarter_end | String | The last day of the quarter where run_date is
located, in the format of yyyyMMdd | 20240331 |
+| run_half_year_begin | String | The first day of the half year where run_date
is located, the format is yyyyMMdd | 20240101 |
+| run_half_year_end | String | The last day of the half year where run_date is
located, the format is yyyyMMdd | 20240630 |
+| run_year_begin | String | The first day of the year where run_date is
located, the format is yyyyMMdd | 20240101 |
+| run_year_end | String | The last day of the year where run_date is located,
the format is yyyyMMdd | 20241231 |
+| run_quarter_begin_std | String | The first day of the quarter where run_date
belongs (standard format), the format is yyyy-MM-dd | 2024-01-01 |
+| run_quarter_end_std | String | The last day of the quarter where run_date
belongs (standard format), the format is yyyy-MM-dd | 2024-03-31 |
+| run_half_year_begin_std | String | The first day of the half year where
run_date is located (standard format), the format is yyyy-MM-dd | 2024-01-01 |
+| run_half_year_end_std | String | The last day of the half year where
run_date is located (standard format), the format is yyyy-MM-dd | 2024-06-30 |
+| run_year_begin_std | String | The first day of the year where run_date is
located (standard format), the format is yyyy-MM-dd | 2024-01-01 |
+| run_year_end_std | String | The last day of the year of run_date (standard
format), the format is yyyy-MM-dd | 2024-12-31 |
+| run_tody_h | String | run_today task running time, yyyyMMddHH | 2024030111 |
+| run_tody_h_std | String | The running time of the run_today task yyyy-MM-dd
HH | 2024-03-01 11 |
+
+details:
+
+1. run_date is a date variable that comes with the core, and supports
user-defined dates. If not specified, it defaults to the day before the current
system time.
+2. Definition of other derived built-in date variables: other date built-in
variables are calculated relative to run_date. Once run_date changes, the
values of other variables will also change automatically. Other date variables
do not support setting initial values and can only be modified by modifying
run_date .
+3. The built-in variables support richer usage scenarios: ${run_date-1} is the
day before run_data; ${run_month_begin-1} is the first day of the previous
month of run_month_begin, where -1 means minus one month.
+
+Take sql as an example:
+```
+--@set run_date=20240229
+select
+"${run_date}" as run_date,
+"${run_date_std}" as run_date_std,
+"${run_today}" as run_today,
+"${run_today_std}" as run_today_std,
+"${run_mon}" as run_mon,
+"${run_mon_std}" as run_mon_std,
+"${run_month_begin}" as run_month_begin,
+"${run_month_begin_std}" as run_month_begin_std,
+"${run_month_now_begin}" as run_month_now_begin,
+"${run_month_now_begin_std}" as run_month_now_begin_std,
+"${run_month_end}" as run_month_end,
+"${run_month_end_std}" as run_month_end_std,
+"${run_month_now_end}" as run_month_now_end,
+"${run_month_now_end_std}" as run_month_now_end_std,
+"${run_quarter_begin}" as run_quarter_begin,
+"${run_quarter_end}" as run_quarter_end,
+"${run_half_year_begin}" as run_half_year_begin,
+"${run_half_year_end}" as run_half_year_end,
+"${run_year_begin}" as run_year_begin,
+"${run_year_end}" as run_year_end,
+"${run_quarter_begin_std}" as run_quarter_begin_std,
+"${run_quarter_end_std}" as run_quarter_end_std,
+"${run_half_year_begin_std}" as run_half_year_begin_std,
+"${run_half_year_end_std}" as run_half_year_end_std,
+"${run_year_begin_std}" as run_year_begin_std,
+"${run_year_end_std}" as run_year_end_std,
+"${run_today_h}" as run_tody_h,
+"${run_today_h_std}" as run_tody_h_std
+
+Compile the replaced content:
+select
+"20240229" as run_date,
+"2024-02-29" as run_date_std,
+"20240301" as run_today,
+"2024-03-01" as run_today_std,
+"202402" as run_mon,
+"2024-02" as run_mon_std,
+"20240201" as run_month_begin,
+"2024-02-01" as run_month_begin_std,
+"20240201" as run_month_now_begin,
+"2024-02-01" as run_month_now_begin_std,
+"20240229" as run_month_end,
+"2024-02-29" as run_month_end_std,
+"20240229" as run_month_now_end,
+"2024-02-29" as run_month_now_end_std,
+"20240101" as run_quarter_begin,
+"20240331" as run_quarter_end,
+"20240101" as run_half_year_begin,
+"20240630" as run_half_year_end,
+"20240101" as run_year_begin,
+"20241231" as run_year_end,
+"2024-01-01" as run_quarter_begin_std,
+"2024-03-31" as run_quarter_end_std,
+"2024-01-01" as run_half_year_begin_std,
+"2024-06-30" as run_half_year_end_std,
+"2024-01-01" as run_year_begin_std,
+"2024-12-31" as run_year_end_std,
+"2024030111" as run_tody_h,
+"2024-03-01 11" as run_tody_h_std
+
+```
+
+### 2.2 Custom variables
+ What are custom variables? User variables that
are defined first and used later. User-defined variables temporarily support
the definition of strings, integers, and floating-point variables. Strings
support the + method, and integers and floating-point numbers support the +-*/
method. User-defined variables do not conflict with the set variable syntax
supported by SparkSQL and HQL itself, but the same name is not allowed. How to
define and use custom variabl [...]
+```
+## Defined in the code, specify before the task code
+sql type definition method:
+--@set f=20.1
+The python/Shell type is defined as follows:
+#@set f=20.1
+Note: Only one line to define a variable is supported
+```
+The use is directly used in the code through ```{varName expression}, such as
${f*2}```
+
+### 2.3 Variable scope
+Custom variables also have a scope in linkis, and the priority is that the
variable defined in the script is greater than the Variable defined in the task
parameter and greater than the built-in run_date variable. The task parameters
are defined as follows:
+```
+## restful
+{
+ "executionContent": {"code": "select \"${f-1}\";", "runType": "sql"},
+ "params": {
+ "variable": {f: "20.1"},
+ "configuration": {
+ "runtime": {
+
"linkis.openlookeng.url":"http://127.0.0.1:9090";
+ }
+ }
+ },
+ "source": {"scriptPath": "file:///mnt/bdp/hadoop/1.sql"},
+ "labels": {
+ "engineType": "spark-2.4.3",
+ "userCreator": "hadoop-IDE"
+ }
+}
+## java SDK
+JobSubmitAction. builder
+ .addExecuteCode(code)
+ .setStartupParams(startupMap)
+ .setUser(user) //submit user
+ .addExecuteUser(user) //execute user
+ .setLabels(labels)
+ .setVariableMap(varMap) //setVar
+ .build
+```
+
+## 3. Date Pattern variable introduction
+
+### 3.1 Features
+
+* Support Pattern format time and users can specify it at will
+* Pattern date variables are calculated relative to run_date
* Support ±y/±M/±d/±H etc.
+* Among them, +- is to operate on the linkis built-in parameter run_date
first, and then replace the pattern field before %. Non-Pattern characters do
not support operation and replacement.
Pattern format comparison table:
@@ -26,7 +194,7 @@ D |Day in year |Number| 189
d |Day in month |Number |10
F |Day of week in month| Number |2
E |Day name in week |Text |Tuesday; Tue
-e |Day number of week (1 = Monday, …, 7 = Sunday) |Number |1
+u |Day number of week (1 = Monday, …, 7 = Sunday) |Number |1
a |Am/pm marker |Text |PM
H |Hour in day (0-23) |Number| 0
k |Hour in day (1-24)| Number |24
@@ -39,15 +207,9 @@ z |Time zone |General time zone |Pacific Standard Time;
PST; GMT-08:00
Z |Time zone |RFC 822 time zone |-0800
X |Time zone |ISO 8601 time zone |-08; -0800; -08:00
-## 2. Overall Design
-The overall design and technical architecture refer to [Linkis Custom
Variables](https://linkis.apache.org/docs/latest/architecture/commons/variable/)
-
-## 3. Function introduction
-* The variable types supported by Linkis are divided into custom variables
(not to be repeated) and system built-in variables. The custom variable date
supports +-.
-* Among them, +- is to perform operation on the built-in parameter run_date of
linkis, and then replace the pattern field before %. Non-pattern characters do
not support operation replacement.
-### 3.1 Examples of built-in variables
-You can define parameters that need to be dynamically rendered according to
your own preferences/business actual situation
+### 3.2 Date Pattern variable example
+You can define the parameters that need dynamic rendering according to your
own preferences/business actual situation
variable | result
--- | ---
@@ -60,121 +222,121 @@ variable | result
&{yyyyMM01%-1M} | 20210301
&{HH%-1H} | 14
-### 3.2 Custom Variable Usage Example
+### 3.3 Date Pattern variable usage examples
* Example 1: sql
```sql
SELECT * FROM hive.tmp.fund_nav_histories
WHERE dt <= DATE_FORMAT(DATE_ADD('day', -1, DATE(Date_parse('&{yyyyMMdd%-1d}',
'%Y%m%d'))), '%Y%m%d')
-````
+```
after rendering
```sql
SELECT * FROM hive.tmp.fund_nav_histories
WHERE dt <= DATE_FORMAT(DATE_ADD('day', -1, DATE(Date_parse('20220705',
'%Y%m%d'))), '%Y%m%d')
-````
+```
* Example 2: shell
```shell
aws s3 ls
s3://***/ads/tmp/dws_member_active_detail_d_20210601_20211231/pt=&{yyyyMMdd%-1d}/
-````
+```
after rendering
```shell
aws s3 ls
s3://***/ads/tmp/dws_member_active_detail_d_20210601_20211231/pt=20220705/
-````
+```
* Example 3: datax json
-````json
+```json
{
- "job": {
- "setting": {
- "speed": {
- "channel": 1
- }
- },
- "content": [
- {
- "reader": {
- "name": "s3reader",
- "parameter": {
- "bucket": "****************",
- "path": [
- "ads/tmp/ccass_tm_announcements/&{yyyyMMdd%-1d}/"
- ],
- "stored": "parquet",
- "compression": "NONE",
- "column": [
- {
- "index": 0,
- "type": "int"
- },
- {
- "index": 1,
- "type": "string",
- "constant": "&{yyyyMMdd%-1d}"
- }
- ]
- }
- },
- "writer": {
- "name": "streamwriter",
- "parameter": {
- "print": true
- }
- }
- }
- ]
- }
+ "job": {
+ "setting": {
+ "speed": {
+ "channel": 1
+ }
+ },
+ "content": [
+ {
+ "reader": {
+ "name": "s3reader",
+ "parameter": {
+ "bucket": "**************",
+ "path": [
+ "ads/tmp/ccass_tm_announcements/&{yyyyMMdd%-1d}/"
+ ],
+ "stored": "parquet",
+ "compression": "NONE",
+ "column": [
+ {
+ "index": 0,
+ "type": "int"
+ },
+ {
+ "index": 1,
+ "type": "string",
+ "constant": "&{yyyyMMdd%-1d}"
+ }
+ ]
+ }
+ },
+ "writer": {
+ "name": "streamwriter",
+ "parameter": {
+ "print": true
+ }
+ }
+ }
+ ]
+ }
}
-````
+```
after rendering
-````json
+```json
{
- "job": {
- "setting": {
- "speed": {
- "channel": 1
- }
- },
- "content": [
- {
- "reader": {
- "name": "s3reader",
- "parameter": {
- "bucket": "****************",
- "path": [
- "ads/tmp/ccass_tm_announcements/20220705/"
- ],
- "stored": "parquet",
- "compression": "NONE",
- "column": [
- {
- "index": 0,
- "type": "int"
- },
- {
- "index": 1,
- "type": "string",
- "constant": "20220705"
- }
- ]
- }
- },
- "writer": {
- "name": "streamwriter",
- "parameter": {
- "print": true
- }
- }
- }
- ]
- }
+ "job": {
+ "setting": {
+ "speed": {
+ "channel": 1
+ }
+ },
+ "content": [
+ {
+ "reader": {
+ "name": "s3reader",
+ "parameter": {
+ "bucket": "**************",
+ "path": [
+ "ads/tmp/ccass_tm_announcements/20220705/"
+ ],
+ "stored": "parquet",
+ "compression": "NONE",
+ "column": [
+ {
+ "index": 0,
+ "type": "int"
+ },
+ {
+ "index": 1,
+ "type": "string",
+ "constant": "20220705"
+ }
+ ]
+ }
+ },
+ "writer": {
+ "name": "streamwriter",
+ "parameter": {
+ "print": true
+ }
+ }
+ }
+ ]
+ }
}
-````
-* Example 4: python
-````python
+```
+* Example 4:python
+```python
print(&{yyyyMMdd%-1d})
-````
+```
after rendering
-````
- 20220705
-````
\ No newline at end of file
+```
+ 20220705
+```
\ No newline at end of file
diff --git a/versioned_docs/version-1.3.1/user-guide/dynamic-variables.md
b/versioned_docs/version-1.3.1/user-guide/dynamic-variables.md
index c901317e1e..77b2d8b5fb 100644
--- a/versioned_docs/version-1.3.1/user-guide/dynamic-variables.md
+++ b/versioned_docs/version-1.3.1/user-guide/dynamic-variables.md
@@ -1,15 +1,184 @@
---
-title: Built In Time Variable
-sidebar_position: 7
+title: built-in time variable
+sidebar_position: 6
---
-## 1. General
-### 1.1 Requirements Background
-Users hope that when writing code, the time format requirements are
ever-changing, and the existing [Linkis custom
variables](https://linkis.apache.org/docs/latest/architecture/commons/variable/)
is currently not enough to support these requirements. In addition, some of
the existing time operation -1 means minus one month, and some minus one day,
which is easy for users to confuse
-### 1.2 Target
-* Other date built-in variables are calculated relative to run_date
-* Support Pattern format time and users can specify at will
+## 1. Overview
+### need
+1. The user hopes that Linkis can provide some public variables and then
replace them during execution. For example, the user runs the same SQL in
batches every day, and needs to specify the partition time of the previous day.
Writing based on SQL will be more complicated if the system provides a run_date
variable It will be very convenient to use.
+2. The user hopes that Linkis supports date pattern calculation, supports
writing variables such as &{YYYY-MM-DD} in the code to calculate time variables
+3. The user wants to define variables by himself, such as setting a float
variable, and then use it in the code
+
+### Target
+1. Support variable replacement of task code
+2. Support custom variables, support users to define custom variables in
scripts and task parameters submitted to Linkis, support simple +, - and other
calculations
+3. Preset system variables: run_date, run_month, run_today and other system
variables
+4. Support date pattern variable, support +, - operation of pattern
+
+The specific technical architecture can refer to:
+[Linkis Custom
Variables](https://linkis.apache.org/docs/latest/architecture/commons/variable/)
+
+## 2 Custom variable function introduction
+ The variable types supported by Linkis are
divided into custom variables and system built-in variables. The internal
variables are pre-defined by Linkis and can be used directly. Then different
variable types support different calculation formats: String supports +,
integers and decimals support +-*/, and dates support +-.
+
+### 2.1 Built-in variables
+Currently supported built-in variables are as follows:
+(The variable values in the following table take 20240229 in leap year as a
special date as an example)
+
+| variable name | variable type | variable meaning | variable value example |
+| ------ | -------- | -------- | ---------- |
+| run\_date | String | Data statistics time (users can set it themselves, the
default setting is the day before the current time), if you execute yesterday’s
data today, it will be yesterday’s time, the format is yyyyMMdd | 20240229 |
+| run\_date\_std | String | Data statistics time (standard date format), if
yesterday’s data is executed today, it is yesterday’s time, the format is
yyyy-MM-dd | 2024-02-29 |
+| run_today | String | The day after run_date (data statistics time), the
format is yyyyMMdd | 20240301 |
+| run_today_std | String | The day after run_date (data statistics time)
(standard format), the format is yyyy-MM-dd | 2024-03-01 |
+| run_mon | String | The month of run_date, the format is yyyyMM | 202402 |
+| run_mon_std | String | The month of run_date (standard format), the format
is yyyy-MM | 2024-02 |
+| run\_month\_begin | String | The first day of the month where the data
statistics time is located, the format is yyyyMMdd | 20240201 |
+| run\_month\_begin\_std | String | The first day of the month where run_date
is located (standard date format), the format is yyyy-MM-dd | 2024-02-01 |
+| run_month_now_begin | String | the first day of the previous month where
run_today is located, the format is yyyyMMdd | 20240201 |
+| run_month_now_begin_std | String | The first day of the previous month where
run_today is located (standard format), the format is yyyy-MM-dd | 2024-02-01 |
+| run\_month\_end | String | The last day of the month where run_date belongs,
in the format of yyyyMMdd | 20240229 |
+| run\_month\_end\_std | String | The last day of the month of run_date
(standard date format), the format is yyyy-MM-dd | 2024-02-29 |
+| run_month_now_end | String | the last day of the previous month where
run_today is located, the format is yyyyMMdd | 20240229 |
+| run_month_now_end_std | String | The last day of the previous month where
run_today is located (standard date format), the format is yyyy-MM-dd |
2024-02-29 |
+| run_quarter_begin | String | The first day of the quarter where run_date
belongs, the format is yyyyMMdd | 20240101 |
+| run_quarter_end | String | The last day of the quarter where run_date is
located, in the format of yyyyMMdd | 20240331 |
+| run_half_year_begin | String | The first day of the half year where run_date
is located, the format is yyyyMMdd | 20240101 |
+| run_half_year_end | String | The last day of the half year where run_date is
located, the format is yyyyMMdd | 20240630 |
+| run_year_begin | String | The first day of the year where run_date is
located, the format is yyyyMMdd | 20240101 |
+| run_year_end | String | The last day of the year where run_date is located,
the format is yyyyMMdd | 20241231 |
+| run_quarter_begin_std | String | The first day of the quarter where run_date
belongs (standard format), the format is yyyy-MM-dd | 2024-01-01 |
+| run_quarter_end_std | String | The last day of the quarter where run_date
belongs (standard format), the format is yyyy-MM-dd | 2024-03-31 |
+| run_half_year_begin_std | String | The first day of the half year where
run_date is located (standard format), the format is yyyy-MM-dd | 2024-01-01 |
+| run_half_year_end_std | String | The last day of the half year where
run_date is located (standard format), the format is yyyy-MM-dd | 2024-06-30 |
+| run_year_begin_std | String | The first day of the year where run_date is
located (standard format), the format is yyyy-MM-dd | 2024-01-01 |
+| run_year_end_std | String | The last day of the year of run_date (standard
format), the format is yyyy-MM-dd | 2024-12-31 |
+| run_tody_h | String | run_today task running time, yyyyMMddHH | 2024030111 |
+| run_tody_h_std | String | The running time of the run_today task yyyy-MM-dd
HH | 2024-03-01 11 |
+
+details:
+
+1. run_date is a date variable that comes with the core, and supports
user-defined dates. If not specified, it defaults to the day before the current
system time.
+2. Definition of other derived built-in date variables: other date built-in
variables are calculated relative to run_date. Once run_date changes, the
values of other variables will also change automatically. Other date variables
do not support setting initial values and can only be modified by modifying
run_date .
+3. The built-in variables support richer usage scenarios: ${run_date-1} is the
day before run_data; ${run_month_begin-1} is the first day of the previous
month of run_month_begin, where -1 means minus one month.
+
+Take sql as an example:
+```
+--@set run_date=20240229
+select
+"${run_date}" as run_date,
+"${run_date_std}" as run_date_std,
+"${run_today}" as run_today,
+"${run_today_std}" as run_today_std,
+"${run_mon}" as run_mon,
+"${run_mon_std}" as run_mon_std,
+"${run_month_begin}" as run_month_begin,
+"${run_month_begin_std}" as run_month_begin_std,
+"${run_month_now_begin}" as run_month_now_begin,
+"${run_month_now_begin_std}" as run_month_now_begin_std,
+"${run_month_end}" as run_month_end,
+"${run_month_end_std}" as run_month_end_std,
+"${run_month_now_end}" as run_month_now_end,
+"${run_month_now_end_std}" as run_month_now_end_std,
+"${run_quarter_begin}" as run_quarter_begin,
+"${run_quarter_end}" as run_quarter_end,
+"${run_half_year_begin}" as run_half_year_begin,
+"${run_half_year_end}" as run_half_year_end,
+"${run_year_begin}" as run_year_begin,
+"${run_year_end}" as run_year_end,
+"${run_quarter_begin_std}" as run_quarter_begin_std,
+"${run_quarter_end_std}" as run_quarter_end_std,
+"${run_half_year_begin_std}" as run_half_year_begin_std,
+"${run_half_year_end_std}" as run_half_year_end_std,
+"${run_year_begin_std}" as run_year_begin_std,
+"${run_year_end_std}" as run_year_end_std,
+"${run_today_h}" as run_tody_h,
+"${run_today_h_std}" as run_tody_h_std
+
+Compile the replaced content:
+select
+"20240229" as run_date,
+"2024-02-29" as run_date_std,
+"20240301" as run_today,
+"2024-03-01" as run_today_std,
+"202402" as run_mon,
+"2024-02" as run_mon_std,
+"20240201" as run_month_begin,
+"2024-02-01" as run_month_begin_std,
+"20240201" as run_month_now_begin,
+"2024-02-01" as run_month_now_begin_std,
+"20240229" as run_month_end,
+"2024-02-29" as run_month_end_std,
+"20240229" as run_month_now_end,
+"2024-02-29" as run_month_now_end_std,
+"20240101" as run_quarter_begin,
+"20240331" as run_quarter_end,
+"20240101" as run_half_year_begin,
+"20240630" as run_half_year_end,
+"20240101" as run_year_begin,
+"20241231" as run_year_end,
+"2024-01-01" as run_quarter_begin_std,
+"2024-03-31" as run_quarter_end_std,
+"2024-01-01" as run_half_year_begin_std,
+"2024-06-30" as run_half_year_end_std,
+"2024-01-01" as run_year_begin_std,
+"2024-12-31" as run_year_end_std,
+"2024030111" as run_tody_h,
+"2024-03-01 11" as run_tody_h_std
+
+```
+
+### 2.2 Custom variables
+ What are custom variables? User variables that
are defined first and used later. User-defined variables temporarily support
the definition of strings, integers, and floating-point variables. Strings
support the + method, and integers and floating-point numbers support the +-*/
method. User-defined variables do not conflict with the set variable syntax
supported by SparkSQL and HQL itself, but the same name is not allowed. How to
define and use custom variabl [...]
+```
+## Defined in the code, specify before the task code
+sql type definition method:
+--@set f=20.1
+The python/Shell type is defined as follows:
+#@set f=20.1
+Note: Only one line to define a variable is supported
+```
+The use is directly used in the code through ```{varName expression}, such as
${f*2}```
+
+### 2.3 Variable scope
+Custom variables also have a scope in linkis, and the priority is that the
variable defined in the script is greater than the Variable defined in the task
parameter and greater than the built-in run_date variable. The task parameters
are defined as follows:
+```
+## restful
+{
+ "executionContent": {"code": "select \"${f-1}\";", "runType": "sql"},
+ "params": {
+ "variable": {f: "20.1"},
+ "configuration": {
+ "runtime": {
+
"linkis.openlookeng.url":"http://127.0.0.1:9090";
+ }
+ }
+ },
+ "source": {"scriptPath": "file:///mnt/bdp/hadoop/1.sql"},
+ "labels": {
+ "engineType": "spark-2.4.3",
+ "userCreator": "hadoop-IDE"
+ }
+}
+## java SDK
+JobSubmitAction. builder
+ .addExecuteCode(code)
+ .setStartupParams(startupMap)
+ .setUser(user) //submit user
+ .addExecuteUser(user) //execute user
+ .setLabels(labels)
+ .setVariableMap(varMap) //setVar
+ .build
+```
+
+## 3. Date Pattern variable introduction
+
+### 3.1 Features
+
+* Support Pattern format time and users can specify it at will
+* Pattern date variables are calculated relative to run_date
* Support ±y/±M/±d/±H etc.
+* Among them, +- is to operate on the linkis built-in parameter run_date
first, and then replace the pattern field before %. Non-Pattern characters do
not support operation and replacement.
Pattern format comparison table:
@@ -25,7 +194,7 @@ D |Day in year |Number| 189
d |Day in month |Number |10
F |Day of week in month| Number |2
E |Day name in week |Text |Tuesday; Tue
-e |Day number of week (1 = Monday, …, 7 = Sunday) |Number |1
+u |Day number of week (1 = Monday, …, 7 = Sunday) |Number |1
a |Am/pm marker |Text |PM
H |Hour in day (0-23) |Number| 0
k |Hour in day (1-24)| Number |24
@@ -38,15 +207,9 @@ z |Time zone |General time zone |Pacific Standard Time;
PST; GMT-08:00
Z |Time zone |RFC 822 time zone |-0800
X |Time zone |ISO 8601 time zone |-08; -0800; -08:00
-## 2. Overall Design
-The overall design and technical architecture refer to [Linkis Custom
Variables](https://linkis.apache.org/docs/latest/architecture/commons/variable/)
-
-## 3. Function introduction
-* The variable types supported by Linkis are divided into custom variables
(not to be repeated) and system built-in variables. The custom variable date
supports +-.
-* Among them, +- is to perform operation on the built-in parameter run_date of
linkis, and then replace the pattern field before %. Non-pattern characters do
not support operation replacement.
-### 3.1 Examples of built-in variables
-You can define parameters that need to be dynamically rendered according to
your own preferences/business actual situation
+### 3.2 Date Pattern variable example
+You can define the parameters that need dynamic rendering according to your
own preferences/business actual situation
variable | result
--- | ---
@@ -59,7 +222,7 @@ variable | result
&{yyyyMM01%-1M} | 20210301
&{HH%-1H} | 14
-### 3.2 Custom Variable Usage Example
+### 3.3 Date Pattern variable usage examples
* Example 1: sql
```sql
@@ -84,96 +247,96 @@ aws s3 ls
s3://***/ads/tmp/dws_member_active_detail_d_20210601_20211231/pt=20220
* Example 3: datax json
```json
{
- "job": {
- "setting": {
- "speed": {
- "channel": 1
- }
- },
- "content": [
- {
- "reader": {
- "name": "s3reader",
- "parameter": {
- "bucket": "****************",
- "path": [
- "ads/tmp/ccass_tm_announcements/&{yyyyMMdd%-1d}/"
- ],
- "stored": "parquet",
- "compression": "NONE",
- "column": [
- {
- "index": 0,
- "type": "int"
- },
- {
- "index": 1,
- "type": "string",
- "constant": "&{yyyyMMdd%-1d}"
- }
- ]
- }
- },
- "writer": {
- "name": "streamwriter",
- "parameter": {
- "print": true
- }
- }
- }
- ]
- }
+ "job": {
+ "setting": {
+ "speed": {
+ "channel": 1
+ }
+ },
+ "content": [
+ {
+ "reader": {
+ "name": "s3reader",
+ "parameter": {
+ "bucket": "**************",
+ "path": [
+ "ads/tmp/ccass_tm_announcements/&{yyyyMMdd%-1d}/"
+ ],
+ "stored": "parquet",
+ "compression": "NONE",
+ "column": [
+ {
+ "index": 0,
+ "type": "int"
+ },
+ {
+ "index": 1,
+ "type": "string",
+ "constant": "&{yyyyMMdd%-1d}"
+ }
+ ]
+ }
+ },
+ "writer": {
+ "name": "streamwriter",
+ "parameter": {
+ "print": true
+ }
+ }
+ }
+ ]
+ }
}
```
after rendering
```json
{
- "job": {
- "setting": {
- "speed": {
- "channel": 1
- }
- },
- "content": [
- {
- "reader": {
- "name": "s3reader",
- "parameter": {
- "bucket": "****************",
- "path": [
- "ads/tmp/ccass_tm_announcements/20220705/"
- ],
- "stored": "parquet",
- "compression": "NONE",
- "column": [
- {
- "index": 0,
- "type": "int"
- },
- {
- "index": 1,
- "type": "string",
- "constant": "20220705"
- }
- ]
- }
- },
- "writer": {
- "name": "streamwriter",
- "parameter": {
- "print": true
- }
- }
- }
- ]
- }
+ "job": {
+ "setting": {
+ "speed": {
+ "channel": 1
+ }
+ },
+ "content": [
+ {
+ "reader": {
+ "name": "s3reader",
+ "parameter": {
+ "bucket": "**************",
+ "path": [
+ "ads/tmp/ccass_tm_announcements/20220705/"
+ ],
+ "stored": "parquet",
+ "compression": "NONE",
+ "column": [
+ {
+ "index": 0,
+ "type": "int"
+ },
+ {
+ "index": 1,
+ "type": "string",
+ "constant": "20220705"
+ }
+ ]
+ }
+ },
+ "writer": {
+ "name": "streamwriter",
+ "parameter": {
+ "print": true
+ }
+ }
+ }
+ ]
+ }
}
```
-* Example 4: python
+* Example 4:python
```python
print(&{yyyyMMdd%-1d})
```
after rendering
```
- 20220705
+ 20220705
```
\ No newline at end of file
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
