This is an automated email from the ASF dual-hosted git repository.
fanjia pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/incubator-seatunnel.git
The following commit(s) were added to refs/heads/dev by this push:
new 8241ec193 [Feature][Log] Integrate slf4j and log4j2 for unified
management logs (#3025)
8241ec193 is described below
commit 8241ec193a77ec639302a7762ebc9349bb4df913
Author: hailin0 <[email protected]>
AuthorDate: Wed Nov 2 16:57:38 2022 +0800
[Feature][Log] Integrate slf4j and log4j2 for unified management logs
(#3025)
* [Feature][Log] Integrate slf4j and log4j2 for unified management logs
Maven dependencys:
- Add slf4j & log4j2 provider & bridges scope into root pom.xml
dependencyManagement
- include: slf4j-api、log4j2、log4j2-slf4j-impl(slf4j provider)、xxx-to-slf4j
- exclude(provided):
logback、log4j1、commons-logging、slf4j-other-provider、log4j2-to-slf4j
- Add slf4j & log4j2 & jcl-over-slf4j into root pom.xml dependencies
- Exclude all logging system packages in maven-sheade-plugin
SeaTunnel engine module:
- Enhanced logging http endpoint to dynamically change log4j2 level
- Config `hazelcast.logging.type: log4j2`
SeaTunnel Dist module:
- Use `maven-assembly-plugin` to copy slf4j & log4j2 & jcl-over-slf4j
packages to `lib/logging` for use by starter shell
Engine Starter module[flink]:
- Use `maven-dependency-plugin` to copy slf4j & log4j2 & jcl-over-slf4j
packages to `target/logging-e2e` for use by E2E testcase
- Add `lib/logging` directory to java classpath in start shell for use by
starter class
- Add jcl-over-slf4j package into shade jar
- Config `config/log4j2-console.properties` on engine-client
- Exclude log4j1 & log4j2 & logback & commons-logging from shade jar
- Fix shell script tail last line text
Engine Starter module[spark]:
- Use `maven-dependency-plugin` to copy slf4j & log4j2 & jcl-over-slf4j
packages to `target/logging-e2e` for use by E2E testcase
- Add `lib/logging` directory to java classpath in start shell for use by
starter class
- Config `config/log4j2-console.properties` on engine-client
- Exclude log4j1 & log4j2 & logback & commons-logging from shade jar
- Fix shell script tail last line text
Engine Starter module[seatunnel]:
- Add `disruptor` jar and config `-Dlog4j2.contextSelector` to use
asynchronous logger on engine-server
- Config `config/log4j2-file.properties` on engine-server
- Config `config/log4j2-console.properties` on engine-client
- Add slf4j & log4j2 & jcl-over-slf4j & log4j-1.2-api package into starter
shade jar
Connector-v1 module[seatunnel-connector-flink-clickhouse]:
- Exclude commons-logging from shade jar
E2E:
- Remove `log4j.properties` files
- Add `log4j2-test.properties` output log to `STDOUT` or `STDERR`
- Copy starter[flink、spark] `target/logging-e2e` into engine container
- Copy config dir into engine container
- Improve execute job logs
* merge dev change
Co-authored-by: Kirs <[email protected]>
---
config/log4j2.properties | 78 +++++++
pom.xml | 223 ++++++++++++++++++---
seatunnel-core/pom.xml | 40 ++--
seatunnel-core/seatunnel-core-base/pom.xml | 21 --
seatunnel-core/seatunnel-core-flink-sql/pom.xml | 43 ++++
.../src/main/bin/start-seatunnel-sql.sh | 15 +-
seatunnel-core/seatunnel-core-flink/pom.xml | 43 ++++
.../src/main/bin/start-seatunnel-flink.sh | 22 +-
seatunnel-core/seatunnel-core-spark/pom.xml | 52 +++++
.../src/main/bin/start-seatunnel-spark.sh | 16 +-
seatunnel-core/seatunnel-core-starter/pom.xml | 21 --
seatunnel-core/seatunnel-flink-starter/pom.xml | 43 ++++
.../main/bin/start-seatunnel-flink-connector-v2.sh | 16 +-
seatunnel-core/seatunnel-spark-starter/pom.xml | 52 +++++
.../main/bin/start-seatunnel-spark-connector-v2.sh | 16 +-
seatunnel-core/seatunnel-starter/pom.xml | 41 ++++
.../src/main/bin/seatunnel-cluster.sh | 10 +
.../seatunnel-starter/src/main/bin/seatunnel.sh | 13 ++
.../src/main/resources/log4j2.properties | 42 ++++
seatunnel-dist/release-docs/LICENSE | 14 +-
seatunnel-dist/release-docs/NOTICE | 19 +-
.../src/main/assembly/assembly-bin-ci.xml | 15 ++
seatunnel-dist/src/main/assembly/assembly-bin.xml | 15 ++
seatunnel-e2e/pom.xml | 5 -
.../src/test/resources/log4j.properties | 22 --
.../src/test/resources/log4j.properties | 22 --
.../src/test/resources/log4j.properties | 22 --
.../src/test/resources/log4j.properties | 22 --
.../common/container/AbstractTestContainer.java | 26 ++-
.../flink/AbstractTestFlinkContainer.java | 1 +
.../spark/AbstractTestSparkContainer.java | 1 +
.../seatunnel/e2e/common/util/ContainerUtil.java | 10 +
.../src/test/resources/log4j.properties | 22 --
.../src/test/resources/log4j2.properties | 42 ++++
.../src/test/resources/log4j.properties | 25 ---
.../src/test/resources/log4j2-test.properties | 47 +++++
.../src/test/resources/log4j.properties | 22 --
.../src/test/resources/log4j.properties | 22 --
.../src/test/resources/log4j.properties | 22 --
.../src/test/resources/log4j.properties | 22 --
.../src/test/resources/log4j.properties | 22 --
.../src/test/resources/log4j.properties | 22 --
.../src/test/resources/log4j.properties | 22 --
.../src/test/resources/log4j.properties | 22 --
.../src/test/resources/log4j.properties | 22 --
.../src/test/resources/log4j.properties | 23 ---
.../src/test/resources/log4j.properties | 23 ---
.../src/test/resources/log4j.properties | 23 ---
.../src/test/resources/log4j.properties | 23 ---
.../src/test/resources/log4j.properties | 23 ---
.../src/test/resources/log4j.properties | 23 ---
.../src/test/resources/log4j.properties | 22 --
.../src/test/resources/log4j.properties | 22 --
.../src/test/resources/log4j.properties | 22 --
.../src/test/resources/log4j.properties | 22 --
.../src/test/resources/log4j.properties | 22 --
.../src/test/resources/log4j.properties | 22 --
.../src/test/resources/log4j.properties | 23 ---
.../src/test/resources/log4j.properties | 23 ---
.../src/test/resources/log4j.properties | 22 --
.../src/test/resources/log4j.properties | 22 --
.../src/test/resources/log4j.properties | 22 --
.../src/test/resources/log4j2-test.properties | 42 ++++
.../src/main/resources/hazelcast-client.yaml | 3 +-
.../src/main/resources/hazelcast.yaml | 6 +
.../seatunnel/engine/server/NodeExtension.java | 17 ++
.../server/log/Log4j2HttpGetCommandProcessor.java | 89 ++++++++
.../server/log/Log4j2HttpPostCommandProcessor.java | 90 +++++++++
.../src/test/resources/log4j2-test.properties | 42 ++++
.../src/test/resources/log4j2-test.properties | 42 ++++
tools/dependencies/known-dependencies.txt | 8 +-
71 files changed, 1215 insertions(+), 841 deletions(-)
diff --git a/config/log4j2.properties b/config/log4j2.properties
new file mode 100644
index 000000000..68ed30982
--- /dev/null
+++ b/config/log4j2.properties
@@ -0,0 +1,78 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+property.file_path = ${sys:seatunnel.logs.path:-/tmp/seatunnel/logs}
+property.file_name = ${sys:seatunnel.logs.file_name:-seatunnel}
+property.file_split_size = 100MB
+property.file_count = 100
+property.file_ttl = 7d
+
+rootLogger.level = INFO
+
+############################ log output to console
#############################
+rootLogger.appenderRef.consoleStdout.ref = consoleStdoutAppender
+rootLogger.appenderRef.consoleStderr.ref = consoleStderrAppender
+############################ log output to console
#############################
+############################ log output to file
#############################
+#rootLogger.appenderRef.file.ref = fileAppender
+############################ log output to file
#############################
+
+appender.consoleStdout.name = consoleStdoutAppender
+appender.consoleStdout.type = CONSOLE
+appender.consoleStdout.target = SYSTEM_OUT
+appender.consoleStdout.layout.type = PatternLayout
+appender.consoleStdout.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %c -
%m%n
+appender.consoleStdout.filter.acceptLtWarn.type = ThresholdFilter
+appender.consoleStdout.filter.acceptLtWarn.level = WARN
+appender.consoleStdout.filter.acceptLtWarn.onMatch = DENY
+appender.consoleStdout.filter.acceptLtWarn.onMismatch = ACCEPT
+
+appender.consoleStderr.name = consoleStderrAppender
+appender.consoleStderr.type = CONSOLE
+appender.consoleStderr.target = SYSTEM_ERR
+appender.consoleStderr.layout.type = PatternLayout
+appender.consoleStderr.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %c -
%m%n
+appender.consoleStderr.filter.acceptGteWarn.type = ThresholdFilter
+appender.consoleStderr.filter.acceptGteWarn.level = WARN
+appender.consoleStderr.filter.acceptGteWarn.onMatch = ACCEPT
+appender.consoleStderr.filter.acceptGteWarn.onMismatch = DENY
+
+appender.file.name = fileAppender
+appender.file.type = RollingFile
+appender.file.fileName = ${file_path}/${file_name}.log
+appender.file.filePattern = ${file_path}/${file_name}.log.%d{yyyy-MM-dd}-%i
+appender.file.append = true
+appender.file.layout.type = PatternLayout
+appender.file.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %c - %m%n
+appender.file.policies.type = Policies
+appender.file.policies.time.type = TimeBasedTriggeringPolicy
+appender.file.policies.time.modulate = true
+appender.file.policies.size.type = SizeBasedTriggeringPolicy
+appender.file.policies.size.size = ${file_split_size}
+appender.file.strategy.type = DefaultRolloverStrategy
+appender.file.strategy.fileIndex = nomax
+appender.file.strategy.action.type = Delete
+appender.file.strategy.action.basepath = ${file_path}
+appender.file.strategy.action.maxDepth = 1
+appender.file.strategy.action.condition.type = IfFileName
+appender.file.strategy.action.condition.glob = ${file_name}.log*
+appender.file.strategy.action.condition.nested_condition.type = IfAny
+appender.file.strategy.action.condition.nested_condition.lastModify.type =
IfLastModified
+appender.file.strategy.action.condition.nested_condition.lastModify.age =
${file_ttl}
+appender.file.strategy.action.condition.nested_condition.fileCount.type =
IfAccumulatedFileCount
+appender.file.strategy.action.condition.nested_condition.fileCount.exceeds =
${file_count}
diff --git a/pom.xml b/pom.xml
index 282e1bc28..4d1785317 100644
--- a/pom.xml
+++ b/pom.xml
@@ -144,11 +144,17 @@
<scala.binary.version>2.11</scala.binary.version>
<maven.compiler.source>${java.version}</maven.compiler.source>
<maven.compiler.target>${java.version}</maven.compiler.target>
+
+ <slf4j.version>1.7.25</slf4j.version>
+ <log4j2.version>2.17.1</log4j2.version>
+ <log4j2-disruptor.version>3.4.4</log4j2-disruptor.version>
+ <log4j.version>1.2.17</log4j.version>
+ <logback.version>1.2.3</logback.version>
+ <commons-logging.version>1.2</commons-logging.version>
<iceberg.version>0.13.1</iceberg.version>
<flink.1.13.6.version>1.13.6</flink.1.13.6.version>
<spark.2.4.0.version>2.4.0</spark.2.4.0.version>
<spark.binary.2.4.version>2.4</spark.binary.2.4.version>
- <commons.logging.version>1.2</commons.logging.version>
<commons.beanutils.version>1.9.4</commons.beanutils.version>
<commons.cli.version>1.4</commons.cli.version>
<commons.configuration.version>1.7</commons.configuration.version>
@@ -198,6 +204,7 @@
<maven-license-maven-plugin>1.20</maven-license-maven-plugin>
<log4j-core.version>2.17.1</log4j-core.version>
<docker-maven-plugin.version>0.38.0</docker-maven-plugin.version>
+
<maven-dependency-plugin.version>3.1.1</maven-dependency-plugin.version>
<p3c-pmd.version>1.3.0</p3c-pmd.version>
<maven-scm-provider-jgit.version>1.9.5</maven-scm-provider-jgit.version>
<testcontainer.version>1.17.3</testcontainer.version>
@@ -205,7 +212,6 @@
<skipUT>false</skipUT>
<skipIT>true</skipIT>
<elasticsearch>7</elasticsearch>
- <slf4j.version>1.7.25</slf4j.version>
<guava.version>19.0</guava.version>
<auto-service.version>1.0.1</auto-service.version>
<hadoop2.version>2.6.5</hadoop2.version>
@@ -219,6 +225,144 @@
<dependencyManagement>
<dependencies>
+ <!-- ***************** slf4j & provider & bridges start
***************** -->
+ <!-- Declare slf4j-api -->
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-api</artifactId>
+ <version>${slf4j.version}</version>
+ </dependency>
+ <!-- Declare slf4j-api provider: log4j2.x -->
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <version>${log4j2.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-api</artifactId>
+ <version>${log4j2.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-core</artifactId>
+ <version>${log4j2.version}</version>
+ </dependency>
+ <!-- Declare log4j2 asynchronous loggers provider: disruptor -->
+ <dependency>
+ <groupId>com.lmax</groupId>
+ <artifactId>disruptor</artifactId>
+ <version>${log4j2-disruptor.version}</version>
+ </dependency>
+ <!-- Include the logging bridges -->
+ <!-- commons-logging bridge to slf4j -->
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <version>${slf4j.version}</version>
+ </dependency>
+ <!-- jdk-logging bridge to slf4j -->
+ <!-- low performance, see:
https://www.slf4j.org/legacy.html#jul-to-slf4j
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jul-to-slf4j</artifactId>
+ <version>${slf4j.version}</version>
+ </dependency>
+ -->
+ <!-- log4j1.x bridge to log4j2.x -->
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-1.2-api</artifactId>
+ <version>${log4j2.version}</version>
+ </dependency>
+ <!-- Exclude the logging bridges via provided scope -->
+ <!-- log4j1.x bridge to slf4j
+ Use of the SLF4J adapter (log4j-over-slf4j) together with the
SLF4J bridge (slf4j-log4j12) should never be attempted as it will cause events
to endlessly be routed between SLF4J and Log4j 1
+ -->
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>log4j-over-slf4j</artifactId>
+ <version>${slf4j.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <!-- slf4j binding to log4j1.x -->
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-log4j12</artifactId>
+ <version>${slf4j.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <!-- log4j2.x binding to slf4j.
+ Use of the SLF4J adapter (log4j-to-slf4j-2.x.jar) together
with the SLF4J bridge (log4j-slf4j-impl-2.x.jar) should never be attempted as
it will cause events to endlessly be routed between SLF4J and Log4j 2
+ -->
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-to-slf4j</artifactId>
+ <version>${log4j2.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <!-- slf4j binding to jdk-logging -->
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-jdk14</artifactId>
+ <version>${slf4j.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <!-- slf4j binding to commons-logging -->
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-jcl</artifactId>
+ <version>${slf4j.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <!-- slf4j binding to nop -->
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-nop</artifactId>
+ <version>${slf4j.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <!-- slf4j binding to simple -->
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-simple</artifactId>
+ <version>${slf4j.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <!-- slf4j binding to reload4j -->
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-reload4j</artifactId>
+ <version>${slf4j.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <!-- Exclude other logging provider via provided scope -->
+ <dependency>
+ <groupId>commons-logging</groupId>
+ <artifactId>commons-logging</artifactId>
+ <version>${commons-logging.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>log4j</groupId>
+ <artifactId>log4j</artifactId>
+ <version>${log4j.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>ch.qos.logback</groupId>
+ <artifactId>logback-classic</artifactId>
+ <version>${logback.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>ch.qos.logback</groupId>
+ <artifactId>logback-core</artifactId>
+ <version>${logback.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <!-- ***************** slf4j & provider & bridges end
***************** -->
+
<dependency>
<groupId>org.apache.seatunnel</groupId>
<artifactId>seatunnel-config-shade</artifactId>
@@ -239,6 +383,10 @@
<groupId>xml-apis</groupId>
<artifactId>xml-apis</artifactId>
</exclusion>
+ <exclusion>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-log4j12</artifactId>
+ </exclusion>
</exclusions>
</dependency>
@@ -315,11 +463,6 @@
</exclusions>
</dependency>
- <dependency>
- <groupId>org.apache.logging.log4j</groupId>
- <artifactId>log4j-core</artifactId>
- <version>${log4j-core.version}</version>
- </dependency>
<dependency>
<groupId>com.typesafe</groupId>
<artifactId>config</artifactId>
@@ -338,24 +481,6 @@
<version>${guava.version}</version>
</dependency>
- <dependency>
- <groupId>org.slf4j</groupId>
- <artifactId>slf4j-api</artifactId>
- <version>${slf4j.version}</version>
- </dependency>
-
- <dependency>
- <groupId>org.slf4j</groupId>
- <artifactId>slf4j-log4j12</artifactId>
- <version>${slf4j.version}</version>
- </dependency>
-
- <dependency>
- <groupId>commons-logging</groupId>
- <artifactId>commons-logging</artifactId>
- <version>${commons.logging.version}</version>
- </dependency>
-
<dependency>
<groupId>org.checkerframework</groupId>
<artifactId>checker-qual</artifactId>
@@ -409,16 +534,34 @@
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</dependency>
+
+ <!-- ***************** slf4j & provider & bridges start
***************** -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
- <version>${slf4j.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-api</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-core</artifactId>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
- <artifactId>slf4j-log4j12</artifactId>
- <version>${slf4j.version}</version>
+ <artifactId>jcl-over-slf4j</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-1.2-api</artifactId>
</dependency>
+ <!-- ***************** slf4j & provider & bridges end
***************** -->
+
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
@@ -463,6 +606,23 @@
<excludes>
<exclude>**/*IT.java</exclude>
</excludes>
+ <classpathDependencyExcludes>
+ <!--
+ The logger provider & bridges declared under
'provided' scope should be explicitly excluded from testing as below.
+ -->
+
<classpathDependencyExclude>org.slf4j:slf4j-jdk14</classpathDependencyExclude>
+
<classpathDependencyExclude>org.slf4j:slf4j-jcl</classpathDependencyExclude>
+
<classpathDependencyExclude>org.slf4j:slf4j-nop</classpathDependencyExclude>
+
<classpathDependencyExclude>org.slf4j:slf4j-simple</classpathDependencyExclude>
+
<classpathDependencyExclude>org.slf4j:slf4j-reload4j</classpathDependencyExclude>
+
<classpathDependencyExclude>org.slf4j:slf4j-log4j12</classpathDependencyExclude>
+
<classpathDependencyExclude>org.slf4j:log4j-over-slf4j</classpathDependencyExclude>
+
<classpathDependencyExclude>commons-logging:commons-logging</classpathDependencyExclude>
+
<classpathDependencyExclude>log4j:log4j</classpathDependencyExclude>
+
<classpathDependencyExclude>ch.qos.logback:logback-classic</classpathDependencyExclude>
+
<classpathDependencyExclude>ch.qos.logback:logback-core</classpathDependencyExclude>
+
<classpathDependencyExclude>org.apache.logging.log4j:log4j-to-slf4j</classpathDependencyExclude>
+ </classpathDependencyExcludes>
</configuration>
</plugin>
<plugin>
@@ -503,6 +663,8 @@
<exclude>org.slf4j:*</exclude>
<exclude>ch.qos.logback:*</exclude>
<exclude>log4j:*</exclude>
+ <exclude>org.apache.logging.log4j:*</exclude>
+ <exclude>commons-logging:*</exclude>
</excludes>
</artifactSet>
<filters>
@@ -670,6 +832,11 @@
</execution>
</executions>
</plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-dependency-plugin</artifactId>
+ <version>${maven-dependency-plugin.version}</version>
+ </plugin>
</plugins>
</pluginManagement>
diff --git a/seatunnel-core/pom.xml b/seatunnel-core/pom.xml
index fbbd64a6e..bd8797aac 100644
--- a/seatunnel-core/pom.xml
+++ b/seatunnel-core/pom.xml
@@ -42,22 +42,28 @@
</modules>
<build>
- <plugins>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-shade-plugin</artifactId>
- <configuration>
- <shadedArtifactAttached>false</shadedArtifactAttached>
-
<createDependencyReducedPom>true</createDependencyReducedPom>
- <!-- Make sure the transitive dependencies are written to
the generated pom under <dependencies> -->
-
<promoteTransitiveDependencies>true</promoteTransitiveDependencies>
- <artifactSet>
- <excludes>
- <exclude>ch.qos.logback:*</exclude>
- </excludes>
- </artifactSet>
- </configuration>
- </plugin>
- </plugins>
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-dependency-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>copy-starter-logging-package-for-e2e</id>
+ <phase>package</phase>
+ <goals>
+ <goal>copy-dependencies</goal>
+ </goals>
+ <configuration>
+ <excludeTransitive>false</excludeTransitive>
+
<includeGroupIds>org.slf4j,org.apache.logging.log4j</includeGroupIds>
+
<includeArtifactIds>slf4j-api,jcl-over-slf4j,log4j-slf4j-impl,log4j-api,log4j-core</includeArtifactIds>
+
<outputDirectory>${project.build.directory}/logging-e2e</outputDirectory>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </pluginManagement>
</build>
</project>
diff --git a/seatunnel-core/seatunnel-core-base/pom.xml
b/seatunnel-core/seatunnel-core-base/pom.xml
index 1a35764eb..7852932d0 100644
--- a/seatunnel-core/seatunnel-core-base/pom.xml
+++ b/seatunnel-core/seatunnel-core-base/pom.xml
@@ -61,25 +61,4 @@
<artifactId>jcommander</artifactId>
</dependency>
</dependencies>
-
- <build>
- <plugins>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-shade-plugin</artifactId>
- <executions>
- <execution>
- <phase>package</phase>
- <goals>
- <goal>shade</goal>
- </goals>
- <!-- base module need skip shading -->
- <configuration>
- <skip>true</skip>
- </configuration>
- </execution>
- </executions>
- </plugin>
- </plugins>
- </build>
</project>
diff --git a/seatunnel-core/seatunnel-core-flink-sql/pom.xml
b/seatunnel-core/seatunnel-core-flink-sql/pom.xml
index 39c93e333..a8bbe3ed7 100644
--- a/seatunnel-core/seatunnel-core-flink-sql/pom.xml
+++ b/seatunnel-core/seatunnel-core-flink-sql/pom.xml
@@ -60,5 +60,48 @@
<build>
<finalName>${project.name}</finalName>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-dependency-plugin</artifactId>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <configuration>
+ <artifactSet>
+ <excludes>
+ <!--
+ not excluded:
+ jcl-over-slf4j(commons-logging to slf4j
bridge)
+
+ Flink server lib already include:
+ slf4j-api
+ log4j-api
+ log4j-core
+ log4j-slf4j-impl
+ log4j-1.2-api
+ -->
+ <exclude>org.slf4j:slf4j-api</exclude>
+ <exclude>org.slf4j:slf4j-jdk14</exclude>
+ <exclude>org.slf4j:slf4j-jcl</exclude>
+ <exclude>org.slf4j:slf4j-nop</exclude>
+ <exclude>org.slf4j:slf4j-simple</exclude>
+ <exclude>org.slf4j:slf4j-reload4j</exclude>
+ <exclude>org.slf4j:slf4j-log4j12</exclude>
+ <exclude>org.slf4j:log4j-over-slf4j</exclude>
+ <exclude>log4j:*</exclude>
+ <exclude>commons-logging:*</exclude>
+ <exclude>ch.qos.logback:*</exclude>
+
<exclude>org.apache.logging.log4j:log4j-api</exclude>
+
<exclude>org.apache.logging.log4j:log4j-core</exclude>
+
<exclude>org.apache.logging.log4j:log4j-slf4j-impl</exclude>
+
<exclude>org.apache.logging.log4j:log4j-1.2-api</exclude>
+
<exclude>org.apache.logging.log4j:log4j-to-slf4j</exclude>
+ </excludes>
+ </artifactSet>
+ </configuration>
+ </plugin>
+ </plugins>
</build>
</project>
diff --git
a/seatunnel-core/seatunnel-core-flink-sql/src/main/bin/start-seatunnel-sql.sh
b/seatunnel-core/seatunnel-core-flink-sql/src/main/bin/start-seatunnel-sql.sh
index 5071144c7..f4ef162f9 100755
---
a/seatunnel-core/seatunnel-core-flink-sql/src/main/bin/start-seatunnel-sql.sh
+++
b/seatunnel-core/seatunnel-core-flink-sql/src/main/bin/start-seatunnel-sql.sh
@@ -50,15 +50,24 @@ else
args=$@
fi
+set +u
+# Log4j2 Config
+if [ -e "${CONF_DIR}/log4j2.properties" ]; then
+ JAVA_OPTS="${JAVA_OPTS}
-Dlog4j2.configurationFile=${CONF_DIR}/log4j2.properties"
+ JAVA_OPTS="${JAVA_OPTS} -Dseatunnel.logs.path=${APP_DIR}/logs"
+ JAVA_OPTS="${JAVA_OPTS} -Dseatunnel.logs.file_name=seatunnel-core-flink-sql"
+fi
+
+CLASS_PATH=${APP_DIR}/starter/logging/*:${APP_JAR}
-CMD=$(java -cp ${APP_JAR} ${APP_MAIN} ${args}) && EXIT_CODE=$? || EXIT_CODE=$?
+CMD=$(java ${JAVA_OPTS} -cp ${CLASS_PATH} ${APP_MAIN} ${args}) && EXIT_CODE=$?
|| EXIT_CODE=$?
if [ ${EXIT_CODE} -eq 234 ]; then
# print usage
echo "${CMD}"
exit 0
elif [ ${EXIT_CODE} -eq 0 ]; then
- echo "Execute SeaTunnel Flink SQL Job: ${CMD}"
- eval ${CMD}
+ echo "Execute SeaTunnel Flink SQL Job: $(echo "${CMD}" | tail -n 1)"
+ eval $(echo "${CMD}" | tail -n 1)
else
echo "${CMD}"
exit ${EXIT_CODE}
diff --git a/seatunnel-core/seatunnel-core-flink/pom.xml
b/seatunnel-core/seatunnel-core-flink/pom.xml
index 4be9326ca..60011f655 100644
--- a/seatunnel-core/seatunnel-core-flink/pom.xml
+++ b/seatunnel-core/seatunnel-core-flink/pom.xml
@@ -106,6 +106,49 @@
<build>
<finalName>${project.name}</finalName>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-dependency-plugin</artifactId>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <configuration>
+ <artifactSet>
+ <excludes>
+ <!--
+ not excluded:
+ jcl-over-slf4j(commons-logging to slf4j
bridge)
+
+ Flink server lib already include:
+ slf4j-api
+ log4j-api
+ log4j-core
+ log4j-slf4j-impl
+ log4j-1.2-api
+ -->
+ <exclude>org.slf4j:slf4j-api</exclude>
+ <exclude>org.slf4j:slf4j-jdk14</exclude>
+ <exclude>org.slf4j:slf4j-jcl</exclude>
+ <exclude>org.slf4j:slf4j-nop</exclude>
+ <exclude>org.slf4j:slf4j-simple</exclude>
+ <exclude>org.slf4j:slf4j-reload4j</exclude>
+ <exclude>org.slf4j:slf4j-log4j12</exclude>
+ <exclude>org.slf4j:log4j-over-slf4j</exclude>
+ <exclude>log4j:*</exclude>
+ <exclude>commons-logging:*</exclude>
+ <exclude>ch.qos.logback:*</exclude>
+
<exclude>org.apache.logging.log4j:log4j-api</exclude>
+
<exclude>org.apache.logging.log4j:log4j-core</exclude>
+
<exclude>org.apache.logging.log4j:log4j-slf4j-impl</exclude>
+
<exclude>org.apache.logging.log4j:log4j-1.2-api</exclude>
+
<exclude>org.apache.logging.log4j:log4j-to-slf4j</exclude>
+ </excludes>
+ </artifactSet>
+ </configuration>
+ </plugin>
+ </plugins>
</build>
<profiles>
diff --git
a/seatunnel-core/seatunnel-core-flink/src/main/bin/start-seatunnel-flink.sh
b/seatunnel-core/seatunnel-core-flink/src/main/bin/start-seatunnel-flink.sh
index 21c2a3c9d..4e9a34c6c 100755
--- a/seatunnel-core/seatunnel-core-flink/src/main/bin/start-seatunnel-flink.sh
+++ b/seatunnel-core/seatunnel-core-flink/src/main/bin/start-seatunnel-flink.sh
@@ -51,23 +51,33 @@ else
args=$@
fi
-ENV_PARAMETERS_OR_ERROR=$(java -cp ${APP_JAR} ${ENV_PARAMETERS_MAIN} ${args})
&& EXIT_CODE=$? || EXIT_CODE=$?
+set +u
+# Log4j2 Config
+if [ -e "${CONF_DIR}/log4j2.properties" ]; then
+ JAVA_OPTS="${JAVA_OPTS}
-Dlog4j2.configurationFile=${CONF_DIR}/log4j2.properties"
+ JAVA_OPTS="${JAVA_OPTS} -Dseatunnel.logs.path=${APP_DIR}/logs"
+ JAVA_OPTS="${JAVA_OPTS} -Dseatunnel.logs.file_name=seatunnel-core-flink"
+fi
+
+CLASS_PATH=${APP_DIR}/starter/logging/*:${APP_JAR}
+
+ENV_PARAMETERS_OR_ERROR=$(java ${JAVA_OPTS} -cp ${CLASS_PATH}
${ENV_PARAMETERS_MAIN} ${args}) && EXIT_CODE=$? || EXIT_CODE=$?
if [ ${EXIT_CODE} -eq 0 ]; then
- echo "Export JVM_ARGS: ${ENV_PARAMETERS_OR_ERROR}"
- export JVM_ARGS="${ENV_PARAMETERS_OR_ERROR}"
+ echo "Export JVM_ARGS: $(echo "${ENV_PARAMETERS_OR_ERROR}" | tail -n 1)"
+ export JVM_ARGS="$(echo "${ENV_PARAMETERS_OR_ERROR}" | tail -n 1)"
else
echo "${ENV_PARAMETERS_OR_ERROR}"
exit ${EXIT_CODE}
fi
-CMD=$(java -cp ${APP_JAR} ${APP_MAIN} ${args}) && EXIT_CODE=$? || EXIT_CODE=$?
+CMD=$(java ${JAVA_OPTS} -cp ${CLASS_PATH} ${APP_MAIN} ${args}) && EXIT_CODE=$?
|| EXIT_CODE=$?
if [ ${EXIT_CODE} -eq 234 ]; then
# print usage
echo "${CMD}"
exit 0
elif [ ${EXIT_CODE} -eq 0 ]; then
- echo "Execute SeaTunnel Flink Job: ${CMD}"
- eval ${CMD}
+ echo "Execute SeaTunnel Flink Job: $(echo "${CMD}" | tail -n 1)"
+ eval $(echo "${CMD}" | tail -n 1)
else
echo "${CMD}"
exit ${EXIT_CODE}
diff --git a/seatunnel-core/seatunnel-core-spark/pom.xml
b/seatunnel-core/seatunnel-core-spark/pom.xml
index fa92953dc..9dbaab569 100644
--- a/seatunnel-core/seatunnel-core-spark/pom.xml
+++ b/seatunnel-core/seatunnel-core-spark/pom.xml
@@ -129,6 +129,58 @@
<build>
<finalName>${project.name}</finalName>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-dependency-plugin</artifactId>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <configuration>
+ <artifactSet>
+ <excludes>
+ <!--
+ Spark(2.x) server lib already include:
+ slf4j-api
+ log4j
+ slf4j-log4j12
+ jul-to-slf4j
+ jcl-over-slf4j
+
+ Spark(3.x) server lib already include:
+ slf4j-api
+ log4j-api
+ log4j-core
+ log4j-slf4j-impl
+ log4j-1.2-api
+ jul-to-slf4j
+ jcl-over-slf4j
+ -->
+ <exclude>org.slf4j:slf4j-api</exclude>
+ <exclude>org.slf4j:slf4j-jdk14</exclude>
+ <exclude>org.slf4j:slf4j-jcl</exclude>
+ <exclude>org.slf4j:slf4j-nop</exclude>
+ <exclude>org.slf4j:slf4j-simple</exclude>
+ <exclude>org.slf4j:slf4j-reload4j</exclude>
+ <exclude>org.slf4j:slf4j-log4j12</exclude>
+ <exclude>org.slf4j:jcl-over-slf4j</exclude>
+ <exclude>org.slf4j:jul-to-slf4j</exclude>
+ <!-- spark2.x use slf4j + log4j1.x -->
+ <exclude>org.slf4j:log4j-over-slf4j</exclude>
+ <exclude>log4j:*</exclude>
+ <exclude>commons-logging:*</exclude>
+ <exclude>ch.qos.logback:*</exclude>
+
<exclude>org.apache.logging.log4j:log4j-api</exclude>
+
<exclude>org.apache.logging.log4j:log4j-core</exclude>
+
<exclude>org.apache.logging.log4j:log4j-slf4j-impl</exclude>
+ <!-- spark3.x use slf4j + log4j2.x -->
+
<exclude>org.apache.logging.log4j:log4j-to-slf4j</exclude>
+ </excludes>
+ </artifactSet>
+ </configuration>
+ </plugin>
+ </plugins>
</build>
<profiles>
diff --git
a/seatunnel-core/seatunnel-core-spark/src/main/bin/start-seatunnel-spark.sh
b/seatunnel-core/seatunnel-core-spark/src/main/bin/start-seatunnel-spark.sh
index c656755a9..9328dfe24 100755
--- a/seatunnel-core/seatunnel-core-spark/src/main/bin/start-seatunnel-spark.sh
+++ b/seatunnel-core/seatunnel-core-spark/src/main/bin/start-seatunnel-spark.sh
@@ -49,14 +49,24 @@ else
args=$@
fi
-CMD=$(java -cp ${APP_JAR} ${APP_MAIN} ${args}) && EXIT_CODE=$? || EXIT_CODE=$?
+set +u
+# Log4j2 Config
+if [ -e "${CONF_DIR}/log4j2.properties" ]; then
+ JAVA_OPTS="${JAVA_OPTS}
-Dlog4j2.configurationFile=${CONF_DIR}/log4j2.properties"
+ JAVA_OPTS="${JAVA_OPTS} -Dseatunnel.logs.path=${APP_DIR}/logs"
+ JAVA_OPTS="${JAVA_OPTS} -Dseatunnel.logs.file_name=seatunnel-core-spark"
+fi
+
+CLASS_PATH=${APP_DIR}/starter/logging/*:${APP_JAR}
+
+CMD=$(java ${JAVA_OPTS} -cp ${CLASS_PATH} ${APP_MAIN} ${args}) && EXIT_CODE=$?
|| EXIT_CODE=$?
if [ ${EXIT_CODE} -eq 234 ]; then
# print usage
echo "${CMD}"
exit 0
elif [ ${EXIT_CODE} -eq 0 ]; then
- echo "Execute SeaTunnel Spark Job: $(echo ${CMD} | tail -n 1)"
- eval $(echo ${CMD} | tail -n 1)
+ echo "Execute SeaTunnel Spark Job: $(echo "${CMD}" | tail -n 1)"
+ eval $(echo "${CMD}" | tail -n 1)
else
echo "${CMD}"
exit ${EXIT_CODE}
diff --git a/seatunnel-core/seatunnel-core-starter/pom.xml
b/seatunnel-core/seatunnel-core-starter/pom.xml
index b000ea0a9..be2484541 100644
--- a/seatunnel-core/seatunnel-core-starter/pom.xml
+++ b/seatunnel-core/seatunnel-core-starter/pom.xml
@@ -60,25 +60,4 @@
<artifactId>jcommander</artifactId>
</dependency>
</dependencies>
-
- <build>
- <plugins>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-shade-plugin</artifactId>
- <executions>
- <execution>
- <phase>package</phase>
- <goals>
- <goal>shade</goal>
- </goals>
- <!-- base module need skip shading -->
- <configuration>
- <skip>true</skip>
- </configuration>
- </execution>
- </executions>
- </plugin>
- </plugins>
- </build>
</project>
diff --git a/seatunnel-core/seatunnel-flink-starter/pom.xml
b/seatunnel-core/seatunnel-flink-starter/pom.xml
index a34426b12..872ec1f30 100644
--- a/seatunnel-core/seatunnel-flink-starter/pom.xml
+++ b/seatunnel-core/seatunnel-flink-starter/pom.xml
@@ -101,6 +101,49 @@
<build>
<finalName>${project.name}</finalName>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-dependency-plugin</artifactId>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <configuration>
+ <artifactSet>
+ <excludes>
+ <!--
+ not excluded:
+ jcl-over-slf4j(commons-logging to slf4j
bridge)
+
+ Flink server lib already include:
+ slf4j-api
+ log4j-api
+ log4j-core
+ log4j-slf4j-impl
+ log4j-1.2-api
+ -->
+ <exclude>org.slf4j:slf4j-api</exclude>
+ <exclude>org.slf4j:slf4j-jdk14</exclude>
+ <exclude>org.slf4j:slf4j-jcl</exclude>
+ <exclude>org.slf4j:slf4j-nop</exclude>
+ <exclude>org.slf4j:slf4j-simple</exclude>
+ <exclude>org.slf4j:slf4j-reload4j</exclude>
+ <exclude>org.slf4j:slf4j-log4j12</exclude>
+ <exclude>org.slf4j:log4j-over-slf4j</exclude>
+ <exclude>log4j:*</exclude>
+ <exclude>commons-logging:*</exclude>
+ <exclude>ch.qos.logback:*</exclude>
+
<exclude>org.apache.logging.log4j:log4j-api</exclude>
+
<exclude>org.apache.logging.log4j:log4j-core</exclude>
+
<exclude>org.apache.logging.log4j:log4j-slf4j-impl</exclude>
+
<exclude>org.apache.logging.log4j:log4j-1.2-api</exclude>
+
<exclude>org.apache.logging.log4j:log4j-to-slf4j</exclude>
+ </excludes>
+ </artifactSet>
+ </configuration>
+ </plugin>
+ </plugins>
</build>
<profiles>
diff --git
a/seatunnel-core/seatunnel-flink-starter/src/main/bin/start-seatunnel-flink-connector-v2.sh
b/seatunnel-core/seatunnel-flink-starter/src/main/bin/start-seatunnel-flink-connector-v2.sh
index e817e1f7d..706776cd4 100755
---
a/seatunnel-core/seatunnel-flink-starter/src/main/bin/start-seatunnel-flink-connector-v2.sh
+++
b/seatunnel-core/seatunnel-flink-starter/src/main/bin/start-seatunnel-flink-connector-v2.sh
@@ -50,14 +50,24 @@ else
args=$@
fi
-CMD=$(java -cp ${APP_JAR} ${APP_MAIN} ${args}) && EXIT_CODE=$? || EXIT_CODE=$?
+set +u
+# Log4j2 Config
+if [ -e "${CONF_DIR}/log4j2.properties" ]; then
+ JAVA_OPTS="${JAVA_OPTS}
-Dlog4j2.configurationFile=${CONF_DIR}/log4j2.properties"
+ JAVA_OPTS="${JAVA_OPTS} -Dseatunnel.logs.path=${APP_DIR}/logs"
+ JAVA_OPTS="${JAVA_OPTS} -Dseatunnel.logs.file_name=seatunnel-flink-starter"
+fi
+
+CLASS_PATH=${APP_DIR}/starter/logging/*:${APP_JAR}
+
+CMD=$(java ${JAVA_OPTS} -cp ${CLASS_PATH} ${APP_MAIN} ${args}) && EXIT_CODE=$?
|| EXIT_CODE=$?
if [ ${EXIT_CODE} -eq 234 ]; then
# print usage
echo "${CMD}"
exit 0
elif [ ${EXIT_CODE} -eq 0 ]; then
- echo "Execute SeaTunnel Flink Job: ${CMD}"
- eval ${CMD}
+ echo "Execute SeaTunnel Flink Job: $(echo "${CMD}" | tail -n 1)"
+ eval $(echo "${CMD}" | tail -n 1)
else
echo "${CMD}"
exit ${EXIT_CODE}
diff --git a/seatunnel-core/seatunnel-spark-starter/pom.xml
b/seatunnel-core/seatunnel-spark-starter/pom.xml
index 91dfbf939..7ff006fac 100644
--- a/seatunnel-core/seatunnel-spark-starter/pom.xml
+++ b/seatunnel-core/seatunnel-spark-starter/pom.xml
@@ -105,6 +105,58 @@
<build>
<finalName>${project.name}</finalName>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-dependency-plugin</artifactId>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <configuration>
+ <artifactSet>
+ <excludes>
+ <!--
+ Spark(2.x) server lib already include:
+ slf4j-api
+ log4j
+ slf4j-log4j12
+ jul-to-slf4j
+ jcl-over-slf4j
+
+ Spark(3.x) server lib already include:
+ slf4j-api
+ log4j-api
+ log4j-core
+ log4j-slf4j-impl
+ log4j-1.2-api
+ jul-to-slf4j
+ jcl-over-slf4j
+ -->
+ <exclude>org.slf4j:slf4j-api</exclude>
+ <exclude>org.slf4j:slf4j-jdk14</exclude>
+ <exclude>org.slf4j:slf4j-jcl</exclude>
+ <exclude>org.slf4j:slf4j-nop</exclude>
+ <exclude>org.slf4j:slf4j-simple</exclude>
+ <exclude>org.slf4j:slf4j-reload4j</exclude>
+ <exclude>org.slf4j:slf4j-log4j12</exclude>
+ <exclude>org.slf4j:jcl-over-slf4j</exclude>
+ <exclude>org.slf4j:jul-to-slf4j</exclude>
+ <!-- spark2.x use slf4j + log4j1.x -->
+ <exclude>org.slf4j:log4j-over-slf4j</exclude>
+ <exclude>log4j:*</exclude>
+ <exclude>commons-logging:*</exclude>
+ <exclude>ch.qos.logback:*</exclude>
+
<exclude>org.apache.logging.log4j:log4j-api</exclude>
+
<exclude>org.apache.logging.log4j:log4j-core</exclude>
+
<exclude>org.apache.logging.log4j:log4j-slf4j-impl</exclude>
+ <!-- spark3.x use slf4j + log4j2.x -->
+
<exclude>org.apache.logging.log4j:log4j-to-slf4j</exclude>
+ </excludes>
+ </artifactSet>
+ </configuration>
+ </plugin>
+ </plugins>
</build>
<!-- todo <profiles>
diff --git
a/seatunnel-core/seatunnel-spark-starter/src/main/bin/start-seatunnel-spark-connector-v2.sh
b/seatunnel-core/seatunnel-spark-starter/src/main/bin/start-seatunnel-spark-connector-v2.sh
index 3fee18158..b830b5b0e 100755
---
a/seatunnel-core/seatunnel-spark-starter/src/main/bin/start-seatunnel-spark-connector-v2.sh
+++
b/seatunnel-core/seatunnel-spark-starter/src/main/bin/start-seatunnel-spark-connector-v2.sh
@@ -50,14 +50,24 @@ else
args=$@
fi
-CMD=$(java -cp ${APP_JAR} ${APP_MAIN} ${args}) && EXIT_CODE=$? || EXIT_CODE=$?
+set +u
+# Log4j2 Config
+if [ -e "${CONF_DIR}/log4j2.properties" ]; then
+ JAVA_OPTS="${JAVA_OPTS}
-Dlog4j2.configurationFile=${CONF_DIR}/log4j2.properties"
+ JAVA_OPTS="${JAVA_OPTS} -Dseatunnel.logs.path=${APP_DIR}/logs"
+ JAVA_OPTS="${JAVA_OPTS} -Dseatunnel.logs.file_name=seatunnel-spark-starter"
+fi
+
+CLASS_PATH=${APP_DIR}/starter/logging/*:${APP_JAR}
+
+CMD=$(java ${JAVA_OPTS} -cp ${CLASS_PATH} ${APP_MAIN} ${args}) && EXIT_CODE=$?
|| EXIT_CODE=$?
if [ ${EXIT_CODE} -eq 234 ]; then
# print usage
echo "${CMD}"
exit 0
elif [ ${EXIT_CODE} -eq 0 ]; then
- echo "Execute SeaTunnel Spark Job: $(echo ${CMD} | tail -n 1)"
- eval $(echo ${CMD} | tail -n 1)
+ echo "Execute SeaTunnel Spark Job: $(echo "${CMD}" | tail -n 1)"
+ eval $(echo "${CMD}" | tail -n 1)
else
echo "${CMD}"
exit ${EXIT_CODE}
diff --git a/seatunnel-core/seatunnel-starter/pom.xml
b/seatunnel-core/seatunnel-starter/pom.xml
index 81dccddaf..3b3e1c384 100644
--- a/seatunnel-core/seatunnel-starter/pom.xml
+++ b/seatunnel-core/seatunnel-starter/pom.xml
@@ -52,10 +52,51 @@
<artifactId>seatunnel-engine-server</artifactId>
<version>${project.version}</version>
</dependency>
+
+ <!-- logger provider & bridges -->
+ <!-- Declare log4j2 asynchronous loggers provider: disruptor -->
+ <dependency>
+ <groupId>com.lmax</groupId>
+ <artifactId>disruptor</artifactId>
+ </dependency>
+ <!-- logger provider & bridges -->
+
</dependencies>
<build>
<finalName>${project.name}</finalName>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <configuration>
+ <artifactSet>
+ <excludes>
+ <!--
+ not excluded:
+ slf4j-api
+ log4j2-api
+ log4j2-core
+ log4j-slf4j-impl
+ log4j-1.2-api(log4j1.x to log4j2.x bridge)
+ jcl-over-slf4j(commons-logging to slf4j
bridge)
+ -->
+ <exclude>org.slf4j:slf4j-jdk14</exclude>
+ <exclude>org.slf4j:slf4j-jcl</exclude>
+ <exclude>org.slf4j:slf4j-nop</exclude>
+ <exclude>org.slf4j:slf4j-simple</exclude>
+ <exclude>org.slf4j:slf4j-reload4j</exclude>
+ <exclude>org.slf4j:slf4j-log4j12</exclude>
+ <exclude>org.slf4j:log4j-over-slf4j</exclude>
+ <exclude>log4j:*</exclude>
+ <exclude>commons-logging:*</exclude>
+ <exclude>ch.qos.logback:*</exclude>
+
<exclude>org.apache.logging.log4j:log4j-to-slf4j</exclude>
+ </excludes>
+ </artifactSet>
+ </configuration>
+ </plugin>
+ </plugins>
</build>
</project>
\ No newline at end of file
diff --git a/seatunnel-core/seatunnel-starter/src/main/bin/seatunnel-cluster.sh
b/seatunnel-core/seatunnel-starter/src/main/bin/seatunnel-cluster.sh
index 4d7758be1..97b2a8f81 100755
--- a/seatunnel-core/seatunnel-starter/src/main/bin/seatunnel-cluster.sh
+++ b/seatunnel-core/seatunnel-starter/src/main/bin/seatunnel-cluster.sh
@@ -63,6 +63,16 @@ fi
JAVA_OPTS="${JAVA_OPTS} -Dseatunnel.config=${SEATUNNEL_CONFIG}"
JAVA_OPTS="${JAVA_OPTS} -Dhazelcast.config=${HAZELCAST_CONFIG}"
+# Log4j2 Config
+JAVA_OPTS="${JAVA_OPTS}
-Dlog4j2.contextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector"
+if [ -e "${CONF_DIR}/log4j2.properties" ]; then
+ JAVA_OPTS="${JAVA_OPTS}
-Dlog4j2.configurationFile=${CONF_DIR}/log4j2.properties"
+ JAVA_OPTS="${JAVA_OPTS} -Dseatunnel.logs.path=${APP_DIR}/logs"
+ JAVA_OPTS="${JAVA_OPTS} -Dseatunnel.logs.file_name=seatunnel-starter-server"
+fi
+
+echo "JAVA_OPTS: ${JAVA_OPTS}"
+
CLASS_PATH=${APP_DIR}/lib/*:${APP_JAR}
java ${JAVA_OPTS} -cp ${CLASS_PATH} ${APP_MAIN} ${args}
diff --git a/seatunnel-core/seatunnel-starter/src/main/bin/seatunnel.sh
b/seatunnel-core/seatunnel-starter/src/main/bin/seatunnel.sh
index 3b0c64820..7bf8fa0a8 100755
--- a/seatunnel-core/seatunnel-starter/src/main/bin/seatunnel.sh
+++ b/seatunnel-core/seatunnel-starter/src/main/bin/seatunnel.sh
@@ -68,6 +68,19 @@ JAVA_OPTS="${JAVA_OPTS}
-Dhazelcast.client.config=${HAZELCAST_CLIENT_CONFIG}"
JAVA_OPTS="${JAVA_OPTS} -Dseatunnel.config=${SEATUNNEL_CONFIG}"
JAVA_OPTS="${JAVA_OPTS} -Dhazelcast.config=${HAZELCAST_CONFIG}"
+# Log4j2 Config
+if [ -e "${CONF_DIR}/log4j2.properties" ]; then
+ JAVA_OPTS="${JAVA_OPTS}
-Dlog4j2.configurationFile=${CONF_DIR}/log4j2.properties"
+ JAVA_OPTS="${JAVA_OPTS} -Dseatunnel.logs.path=${APP_DIR}/logs"
+ if [[ $args == *" -e local"* || $args == *" --deploy-mode local"* ]]; then
+ JAVA_OPTS="${JAVA_OPTS}
-Dseatunnel.logs.file_name=seatunnel-starter-client-$((`date '+%s'`*1000+`date
'+%N'`/1000000))"
+ else
+ JAVA_OPTS="${JAVA_OPTS}
-Dseatunnel.logs.file_name=seatunnel-starter-client"
+ fi
+fi
+
+echo "JAVA_OPTS: ${JAVA_OPTS}"
+
CLASS_PATH=${APP_DIR}/lib/*:${APP_JAR}
java ${JAVA_OPTS} -cp ${CLASS_PATH} ${APP_MAIN} ${args}
diff --git
a/seatunnel-core/seatunnel-starter/src/main/resources/log4j2.properties
b/seatunnel-core/seatunnel-starter/src/main/resources/log4j2.properties
new file mode 100644
index 000000000..2dc1b8ca5
--- /dev/null
+++ b/seatunnel-core/seatunnel-starter/src/main/resources/log4j2.properties
@@ -0,0 +1,42 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+rootLogger.level = INFO
+
+rootLogger.appenderRef.consoleStdout.ref = consoleStdoutAppender
+rootLogger.appenderRef.consoleStderr.ref = consoleStderrAppender
+
+appender.consoleStdout.name = consoleStdoutAppender
+appender.consoleStdout.type = CONSOLE
+appender.consoleStdout.target = SYSTEM_OUT
+appender.consoleStdout.layout.type = PatternLayout
+appender.consoleStdout.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %c -
%m%n
+appender.consoleStdout.filter.acceptLtWarn.type = ThresholdFilter
+appender.consoleStdout.filter.acceptLtWarn.level = WARN
+appender.consoleStdout.filter.acceptLtWarn.onMatch = DENY
+appender.consoleStdout.filter.acceptLtWarn.onMismatch = ACCEPT
+
+appender.consoleStderr.name = consoleStderrAppender
+appender.consoleStderr.type = CONSOLE
+appender.consoleStderr.target = SYSTEM_ERR
+appender.consoleStderr.layout.type = PatternLayout
+appender.consoleStderr.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %c -
%m%n
+appender.consoleStderr.filter.acceptGteWarn.type = ThresholdFilter
+appender.consoleStderr.filter.acceptGteWarn.level = WARN
+appender.consoleStderr.filter.acceptGteWarn.onMatch = ACCEPT
+appender.consoleStderr.filter.acceptGteWarn.onMismatch = DENY
diff --git a/seatunnel-dist/release-docs/LICENSE
b/seatunnel-dist/release-docs/LICENSE
index e6fa2de77..9aed11646 100644
--- a/seatunnel-dist/release-docs/LICENSE
+++ b/seatunnel-dist/release-docs/LICENSE
@@ -277,7 +277,10 @@ The text of each license is the standard Apache 2.0
license.
(The Apache Software License, Version 2.0) Data Mapper for Jackson
(org.codehaus.jackson:jackson-mapper-asl:1.9.13 - http://jackson.codehaus.org)
(Apache License, Version 2.0) jcommander (com.beust:jcommander:1.81 -
https://jcommander.org)
(The Apache Software License, Version 2.0) FindBugs-jsr305
(com.google.code.findbugs:jsr305:1.3.9 - http://findbugs.sourceforge.net/)
- (The Apache Software License, Version 2.0) Apache Log4j
(log4j:log4j:1.2.17 - http://logging.apache.org/log4j/1.2/)
+ (The Apache Software License, Version 2.0) Apache Log4j
(org.apache.logging.log4j:log4j-api:2.17.1 -
https://logging.apache.org/log4j/2.x/)
+ (The Apache Software License, Version 2.0) Apache Log4j
(org.apache.logging.log4j:log4j-core:2.17.1 -
https://logging.apache.org/log4j/2.x/)
+ (The Apache Software License, Version 2.0) Apache Log4j
(org.apache.logging.log4j:log4j-slf4j-impl:2.17.1 -
https://logging.apache.org/log4j/2.x/)
+ (The Apache Software License, Version 2.0) Apache Log4j
(org.apache.logging.log4j:log4j-1.2-api:2.17.1 -
https://logging.apache.org/log4j/2.x/)
(The Apache Software License, Version 2.0) LZ4 and xxHash
(net.jpountz.lz4:lz4:1.3.0 - https://github.com/jpountz/lz4-java)
(The Apache Software License, Version 2.0) LZ4 and xxHash
(org.lz4:lz4-java:1.6.0 - https://github.com/lz4/lz4-java)
(Apache 2) Objenesis (org.objenesis:objenesis:2.1 - http://objenesis.org)
@@ -301,12 +304,13 @@ The text of each license is the standard Apache 2.0
license.
(Apache-2.0) snappy-java (org.xerial.snappy:snappy-java:1.1.8.3 -
https://github.com/xerial/snappy-java)
(Apache-2.0) snappy-java (org.xerial.snappy:snappy-java:1.1.1.3 -
https://github.com/xerial/snappy-java)
(Apache-2.0) maven-wrapper (org.apache.maven:maven-wrapper:3.8.4
https://maven.apache.org/wrapper/)
- (The Apache Software License, Version 2.0) protostuff
(io.protostuff:protostuff-collectionschema:1.8.0 -
https://github.com/protostuff/protostuff)
+ (The Apache Software License, Version 2.0) protostuff
(io.protostuff:protostuff-collectionschema:1.8.0 -
https://github.com/protostuff/protostuff)
(The Apache Software License, Version 2.0) protostuff
(io.protostuff:protostuff-core:1.8.0 - https://github.com/protostuff/protostuff)
(The Apache Software License, Version 2.0) protostuff
(io.protostuff:protostuff-api:1.8.0 - https://github.com/protostuff/protostuff)
(The Apache Software License, Version 2.0) protostuff
(io.protostuff:protostuff-runtime:1.8.0 -
https://github.com/protostuff/protostuff)
(The Apache Software License, Version 2.0) hazelcast
(com.hazelcast:hazelcast:5.1 - https://github.com/hazelcast/hazelcast)
-
+ (Apache-2.0) disruptor (com.lmax:disruptor:3.4.4
https://lmax-exchange.github.io/disruptor/)
+
========================================================================
MOZILLA PUBLIC LICENSE License
========================================================================
@@ -359,8 +363,8 @@ The following components are provided under the MIT
License. See project link fo
The text of each license is also included at licenses/LICENSE-[project].txt.
(MIT License) scopt (com.github.scopt:scopt_2.11:3.5.0 -
https://github.com/scopt/scopt)
- (MIT License) SLF4J API Module (org.slf4j:slf4j-api:1.7.25 -
http://www.slf4j.org)
- (MIT License) SLF4J LOG4J-12 Binding (org.slf4j:slf4j-log4j12:1.7.25 -
http://www.slf4j.org)
+ (MIT License) slf4j-api (org.slf4j:slf4j-api:1.7.25 -
http://www.slf4j.org)
+ (MIT License) jcl-over-slf4j (org.slf4j:jcl-over-slf4j:1.7.25 -
http://www.slf4j.org)
========================================================================
Public Domain License
========================================================================
diff --git a/seatunnel-dist/release-docs/NOTICE
b/seatunnel-dist/release-docs/NOTICE
index 19c7f5a64..b004e9281 100644
--- a/seatunnel-dist/release-docs/NOTICE
+++ b/seatunnel-dist/release-docs/NOTICE
@@ -464,11 +464,26 @@ from the source code management (SCM) system project uses.
Apache log4j NOTICE
========================================================================
-Apache log4j
-Copyright 2010 The Apache Software Foundation
+Apache Log4j
+Copyright 1999-2021 Apache Software Foundation
This product includes software developed at
The Apache Software Foundation (http://www.apache.org/).
+
+ResolverUtil.java
+Copyright 2005-2006 Tim Fennell
+
+Dumbster SMTP test server
+Copyright 2004 Jason Paul Kitchen
+
+TypeUtil.java
+Copyright 2002-2012 Ramnivas Laddad, Juergen Hoeller, Chris Beams
+
+picocli (http://picocli.info)
+Copyright 2017 Remko Popma
+
+TimeoutBlockingWaitStrategy.java and parts of Util.java
+Copyright 2011 LMAX Ltd.
========================================================================
Apache ORC NOTICE
diff --git a/seatunnel-dist/src/main/assembly/assembly-bin-ci.xml
b/seatunnel-dist/src/main/assembly/assembly-bin-ci.xml
index b197b51a2..ccedb58ee 100644
--- a/seatunnel-dist/src/main/assembly/assembly-bin-ci.xml
+++ b/seatunnel-dist/src/main/assembly/assembly-bin-ci.xml
@@ -115,6 +115,21 @@
</file>
</files>
<dependencySets>
+ <!-- ============ Logging Jars ============ -->
+ <dependencySet>
+ <useProjectArtifact>false</useProjectArtifact>
+ <useTransitiveDependencies>true</useTransitiveDependencies>
+ <unpack>false</unpack>
+ <outputDirectory>/starter/logging</outputDirectory>
+ <includes>
+ <include>org.slf4j:slf4j-api:jar</include>
+ <include>org.slf4j:jcl-over-slf4j:jar</include>
+ <include>org.apache.logging.log4j:log4j-api:jar</include>
+ <include>org.apache.logging.log4j:log4j-core:jar</include>
+
<include>org.apache.logging.log4j:log4j-slf4j-impl:jar</include>
+ </includes>
+ </dependencySet>
+
<!-- ============ Starter Jars ============ -->
<dependencySet>
<useProjectArtifact>false</useProjectArtifact>
diff --git a/seatunnel-dist/src/main/assembly/assembly-bin.xml
b/seatunnel-dist/src/main/assembly/assembly-bin.xml
index a535d48c2..747b28b8e 100644
--- a/seatunnel-dist/src/main/assembly/assembly-bin.xml
+++ b/seatunnel-dist/src/main/assembly/assembly-bin.xml
@@ -135,6 +135,21 @@
</files>
<dependencySets>
+ <!-- ============ Logging Jars ============ -->
+ <dependencySet>
+ <useProjectArtifact>false</useProjectArtifact>
+ <useTransitiveDependencies>true</useTransitiveDependencies>
+ <unpack>false</unpack>
+ <outputDirectory>/starter/logging</outputDirectory>
+ <includes>
+ <include>org.slf4j:slf4j-api:jar</include>
+ <include>org.slf4j:jcl-over-slf4j:jar</include>
+ <include>org.apache.logging.log4j:log4j-api:jar</include>
+ <include>org.apache.logging.log4j:log4j-core:jar</include>
+
<include>org.apache.logging.log4j:log4j-slf4j-impl:jar</include>
+ </includes>
+ </dependencySet>
+
<!-- ============ Starter Jars ============ -->
<dependencySet>
<useProjectArtifact>false</useProjectArtifact>
diff --git a/seatunnel-e2e/pom.xml b/seatunnel-e2e/pom.xml
index 4a5c70ed5..7ff89647c 100644
--- a/seatunnel-e2e/pom.xml
+++ b/seatunnel-e2e/pom.xml
@@ -47,11 +47,6 @@
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers</artifactId>
</dependency>
- <dependency>
- <groupId>org.slf4j</groupId>
- <artifactId>slf4j-log4j12</artifactId>
- <scope>test</scope>
- </dependency>
<!-- Testcontainers 1.x is tightly coupled with the JUnit 4.x rule
API-->
<dependency>
<groupId>junit</groupId>
diff --git
a/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-assert-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-assert-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index db5d9e512..000000000
---
a/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-assert-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-file-local-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-file-local-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index db5d9e512..000000000
---
a/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-file-local-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index db5d9e512..000000000
---
a/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-redis-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-redis-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index db5d9e512..000000000
---
a/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-redis-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/container/AbstractTestContainer.java
b/seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/container/AbstractTestContainer.java
index 2efeeb058..d7271dd80 100644
---
a/seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/container/AbstractTestContainer.java
+++
b/seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/container/AbstractTestContainer.java
@@ -79,6 +79,12 @@ public abstract class AbstractTestContainer implements
TestContainer {
SEATUNNEL_HOME);
}
+ protected void copySeaTunnelStarterLoggingToContainer(GenericContainer<?>
container) {
+ ContainerUtil.copySeaTunnelStarterLoggingToContainer(container,
+ this.startModuleFullPath,
+ SEATUNNEL_HOME);
+ }
+
protected Container.ExecResult executeJob(GenericContainer<?> container,
String confFile) throws IOException, InterruptedException {
final String confInContainerPath =
copyConfigFileToContainer(container, confFile);
// copy connectors
@@ -100,18 +106,26 @@ public abstract class AbstractTestContainer implements
TestContainer {
command.add(adaptPathForWin(configPath));
command.addAll(getExtraStartShellCommands());
+ LOG.info("Execute config file: {} to Container[{}] "
+ + "\n==================== Shell Command start
====================\n"
+ + "{}"
+ + "\n==================== Shell Command end
====================",
+ configPath, container.getDockerImageName(), String.join(" ",
command));
Container.ExecResult execResult = container.execInContainer("bash",
"-c", String.join(" ", command));
+
if (execResult.getStdout() != null && execResult.getStdout().length()
> 0) {
- LOG.info("\n==================== ExecuteConfigFile: {} STDOUT
start ====================\n"
+ LOG.info("Execute config file: {} to Container[{}] STDOUT:"
+ + "\n==================== STDOUT start
====================\n"
+ "{}"
- + "\n==================== ExecuteConfigFile: {} STDOUT end
====================",
- configPath, execResult.getStdout(), configPath);
+ + "\n==================== STDOUT end
====================",
+ configPath, container.getDockerImageName(),
execResult.getStdout());
}
if (execResult.getStderr() != null && execResult.getStderr().length()
> 0) {
- LOG.error("\n==================== ExecuteConfigFile: {} STDERR
start ====================\n"
+ LOG.error("Execute config file: {} to Container[{}] STDERR:"
+ + "\n==================== STDERR start
====================\n"
+ "{}"
- + "\n==================== ExecuteConfigFile: {} STDERR end
====================",
- configPath, execResult.getStderr(), configPath);
+ + "\n==================== STDERR end
====================",
+ configPath, container.getDockerImageName(),
execResult.getStderr());
}
return execResult;
}
diff --git
a/seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/container/flink/AbstractTestFlinkContainer.java
b/seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/container/flink/AbstractTestFlinkContainer.java
index 4a4dbfa9e..089f77bc2 100644
---
a/seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/container/flink/AbstractTestFlinkContainer.java
+++
b/seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/container/flink/AbstractTestFlinkContainer.java
@@ -75,6 +75,7 @@ public abstract class AbstractTestFlinkContainer extends
AbstractTestContainer {
.withRegEx(".*Starting the resource manager.*")
.withStartupTimeout(Duration.ofMinutes(2)));
copySeaTunnelStarterToContainer(jobManager);
+ copySeaTunnelStarterLoggingToContainer(jobManager);
taskManager = new GenericContainer<>(dockerImage)
.withCommand("taskmanager")
diff --git
a/seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/container/spark/AbstractTestSparkContainer.java
b/seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/container/spark/AbstractTestSparkContainer.java
index ee0fa9cf1..2489ef2c6 100644
---
a/seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/container/spark/AbstractTestSparkContainer.java
+++
b/seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/container/spark/AbstractTestSparkContainer.java
@@ -57,6 +57,7 @@ public abstract class AbstractTestSparkContainer extends
AbstractTestContainer {
.withRegEx(".*Master: Starting Spark master at.*")
.withStartupTimeout(Duration.ofMinutes(2)));
copySeaTunnelStarterToContainer(master);
+ copySeaTunnelStarterLoggingToContainer(master);
// In most case we can just use standalone mode to execute a spark
job, if we want to use cluster mode, we need to
// start a worker.
diff --git
a/seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/util/ContainerUtil.java
b/seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/util/ContainerUtil.java
index 2ff6518b5..6c762844c 100644
---
a/seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/util/ContainerUtil.java
+++
b/seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/util/ContainerUtil.java
@@ -92,6 +92,16 @@ public final class ContainerUtil {
return targetConfInContainer;
}
+ public static void
copySeaTunnelStarterLoggingToContainer(GenericContainer<?> container,
+ String
startModulePath,
+ String
seatunnelHomeInContainer) {
+ // copy logging lib
+ final String loggingLibPath = startModulePath + File.separator +
"target" + File.separator + "logging-e2e" + File.separator;
+ checkPathExist(loggingLibPath);
+
container.withCopyFileToContainer(MountableFile.forHostPath(loggingLibPath),
+ Paths.get(seatunnelHomeInContainer, "starter",
"logging").toString());
+ }
+
public static void copySeaTunnelStarterToContainer(GenericContainer<?>
container,
String startModuleName,
String startModulePath,
diff --git
a/seatunnel-e2e/seatunnel-e2e-common/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-e2e-common/src/test/resources/log4j.properties
deleted file mode 100644
index db5d9e512..000000000
--- a/seatunnel-e2e/seatunnel-e2e-common/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-e2e-common/src/test/resources/log4j2.properties
b/seatunnel-e2e/seatunnel-e2e-common/src/test/resources/log4j2.properties
new file mode 100644
index 000000000..2dc1b8ca5
--- /dev/null
+++ b/seatunnel-e2e/seatunnel-e2e-common/src/test/resources/log4j2.properties
@@ -0,0 +1,42 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+rootLogger.level = INFO
+
+rootLogger.appenderRef.consoleStdout.ref = consoleStdoutAppender
+rootLogger.appenderRef.consoleStderr.ref = consoleStderrAppender
+
+appender.consoleStdout.name = consoleStdoutAppender
+appender.consoleStdout.type = CONSOLE
+appender.consoleStdout.target = SYSTEM_OUT
+appender.consoleStdout.layout.type = PatternLayout
+appender.consoleStdout.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %c -
%m%n
+appender.consoleStdout.filter.acceptLtWarn.type = ThresholdFilter
+appender.consoleStdout.filter.acceptLtWarn.level = WARN
+appender.consoleStdout.filter.acceptLtWarn.onMatch = DENY
+appender.consoleStdout.filter.acceptLtWarn.onMismatch = ACCEPT
+
+appender.consoleStderr.name = consoleStderrAppender
+appender.consoleStderr.type = CONSOLE
+appender.consoleStderr.target = SYSTEM_ERR
+appender.consoleStderr.layout.type = PatternLayout
+appender.consoleStderr.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %c -
%m%n
+appender.consoleStderr.filter.acceptGteWarn.type = ThresholdFilter
+appender.consoleStderr.filter.acceptGteWarn.level = WARN
+appender.consoleStderr.filter.acceptGteWarn.onMatch = ACCEPT
+appender.consoleStderr.filter.acceptGteWarn.onMismatch = DENY
diff --git
a/seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base/src/test/resources/log4j.properties
deleted file mode 100644
index 8757e3b86..000000000
---
a/seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,25 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
-
-log4j.logger.org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator=DEBUG
-log4j.logger.org.apache.seatunnel.engine.server.scheduler.PipelineBaseScheduler=DEBUG
diff --git
a/seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base/src/test/resources/log4j2-test.properties
b/seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base/src/test/resources/log4j2-test.properties
new file mode 100644
index 000000000..eab88a7c7
--- /dev/null
+++
b/seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base/src/test/resources/log4j2-test.properties
@@ -0,0 +1,47 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+rootLogger.level = INFO
+
+rootLogger.appenderRef.consoleStdout.ref = consoleStdoutAppender
+rootLogger.appenderRef.consoleStderr.ref = consoleStderrAppender
+
+logger.checkpoint.name=org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator
+logger.checkpoint.level=DEBUG
+logger.pipeline.name=org.apache.seatunnel.engine.server.scheduler.PipelineBaseScheduler
+logger.pipeline.level=DEBUG
+
+appender.consoleStdout.name = consoleStdoutAppender
+appender.consoleStdout.type = CONSOLE
+appender.consoleStdout.target = SYSTEM_OUT
+appender.consoleStdout.layout.type = PatternLayout
+appender.consoleStdout.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %c -
%m%n
+appender.consoleStdout.filter.acceptLtWarn.type = ThresholdFilter
+appender.consoleStdout.filter.acceptLtWarn.level = WARN
+appender.consoleStdout.filter.acceptLtWarn.onMatch = DENY
+appender.consoleStdout.filter.acceptLtWarn.onMismatch = ACCEPT
+
+appender.consoleStderr.name = consoleStderrAppender
+appender.consoleStderr.type = CONSOLE
+appender.consoleStderr.target = SYSTEM_ERR
+appender.consoleStderr.layout.type = PatternLayout
+appender.consoleStderr.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %c -
%m%n
+appender.consoleStderr.filter.acceptGteWarn.type = ThresholdFilter
+appender.consoleStderr.filter.acceptGteWarn.level = WARN
+appender.consoleStderr.filter.acceptGteWarn.onMatch = ACCEPT
+appender.consoleStderr.filter.acceptGteWarn.onMismatch = DENY
diff --git
a/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-assert-flink-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-assert-flink-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index db5d9e512..000000000
---
a/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-assert-flink-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-datahub-flink-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-datahub-flink-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index db5d9e512..000000000
---
a/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-datahub-flink-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-fake-flink-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-fake-flink-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index db5d9e512..000000000
---
a/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-fake-flink-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-iceberg-flink-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-iceberg-flink-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index db5d9e512..000000000
---
a/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-iceberg-flink-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-iotdb-flink-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-iotdb-flink-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index db5d9e512..000000000
---
a/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-iotdb-flink-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-jdbc-flink-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-jdbc-flink-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index db5d9e512..000000000
---
a/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-jdbc-flink-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-kafka-flink-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-kafka-flink-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index db5d9e512..000000000
---
a/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-kafka-flink-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-mongodb-flink-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-mongodb-flink-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index db5d9e512..000000000
---
a/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-mongodb-flink-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-neo4j-flink-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-neo4j-flink-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index db5d9e512..000000000
---
a/seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-neo4j-flink-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-flink-e2e/seatunnel-connector-flink-assert-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-flink-e2e/seatunnel-connector-flink-assert-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index 57b61a3c1..000000000
---
a/seatunnel-e2e/seatunnel-flink-e2e/seatunnel-connector-flink-assert-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,23 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-# Set everything to be logged to the console
-log4j.rootCategory=ERROR, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-flink-e2e/seatunnel-connector-flink-clickhouse-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-flink-e2e/seatunnel-connector-flink-clickhouse-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index 57b61a3c1..000000000
---
a/seatunnel-e2e/seatunnel-flink-e2e/seatunnel-connector-flink-clickhouse-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,23 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-# Set everything to be logged to the console
-log4j.rootCategory=ERROR, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-flink-e2e/seatunnel-connector-flink-fake-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-flink-e2e/seatunnel-connector-flink-fake-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index 57b61a3c1..000000000
---
a/seatunnel-e2e/seatunnel-flink-e2e/seatunnel-connector-flink-fake-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,23 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-# Set everything to be logged to the console
-log4j.rootCategory=ERROR, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-flink-e2e/seatunnel-connector-flink-file-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-flink-e2e/seatunnel-connector-flink-file-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index 57b61a3c1..000000000
---
a/seatunnel-e2e/seatunnel-flink-e2e/seatunnel-connector-flink-file-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,23 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-# Set everything to be logged to the console
-log4j.rootCategory=ERROR, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-flink-e2e/seatunnel-connector-flink-http-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-flink-e2e/seatunnel-connector-flink-http-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index 57b61a3c1..000000000
---
a/seatunnel-e2e/seatunnel-flink-e2e/seatunnel-connector-flink-http-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,23 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-# Set everything to be logged to the console
-log4j.rootCategory=ERROR, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-flink-sql-e2e/setunnel-connector-flink-sql-fake-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-flink-sql-e2e/setunnel-connector-flink-sql-fake-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index 57b61a3c1..000000000
---
a/seatunnel-e2e/seatunnel-flink-sql-e2e/setunnel-connector-flink-sql-fake-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,23 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-# Set everything to be logged to the console
-log4j.rootCategory=ERROR, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-datahub-spark-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-datahub-spark-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index db5d9e512..000000000
---
a/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-datahub-spark-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-fake-spark-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-fake-spark-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index db5d9e512..000000000
---
a/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-fake-spark-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-iceberg-spark-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-iceberg-spark-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index db5d9e512..000000000
---
a/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-iceberg-spark-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-iotdb-spark-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-iotdb-spark-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index db5d9e512..000000000
---
a/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-iotdb-spark-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-jdbc-spark-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-jdbc-spark-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index db5d9e512..000000000
---
a/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-jdbc-spark-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-kafka-spark-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-kafka-spark-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index db5d9e512..000000000
---
a/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-kafka-spark-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-mongodb-spark-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-mongodb-spark-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index 89ed3ad31..000000000
---
a/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-mongodb-spark-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,23 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-neo4j-spark-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-neo4j-spark-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index 89ed3ad31..000000000
---
a/seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-neo4j-spark-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,23 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-# Set everything to be logged to the console
-log4j.rootCategory=INFO, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-spark-e2e/seatunnel-connector-spark-fake-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-spark-e2e/seatunnel-connector-spark-fake-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index 89547981c..000000000
---
a/seatunnel-e2e/seatunnel-spark-e2e/seatunnel-connector-spark-fake-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=ERROR, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-spark-e2e/seatunnel-connector-spark-http-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-spark-e2e/seatunnel-connector-spark-http-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index 89547981c..000000000
---
a/seatunnel-e2e/seatunnel-spark-e2e/seatunnel-connector-spark-http-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=ERROR, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-e2e/seatunnel-spark-e2e/seatunnel-connector-spark-jdbc-e2e/src/test/resources/log4j.properties
b/seatunnel-e2e/seatunnel-spark-e2e/seatunnel-connector-spark-jdbc-e2e/src/test/resources/log4j.properties
deleted file mode 100644
index 89547981c..000000000
---
a/seatunnel-e2e/seatunnel-spark-e2e/seatunnel-connector-spark-jdbc-e2e/src/test/resources/log4j.properties
+++ /dev/null
@@ -1,22 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Set everything to be logged to the console
-log4j.rootCategory=ERROR, console
-log4j.appender.console=org.apache.log4j.ConsoleAppender
-log4j.appender.console.target=System.err
-log4j.appender.console.layout=org.apache.log4j.PatternLayout
-log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{1}: %m%n
diff --git
a/seatunnel-engine/seatunnel-engine-client/src/test/resources/log4j2-test.properties
b/seatunnel-engine/seatunnel-engine-client/src/test/resources/log4j2-test.properties
new file mode 100644
index 000000000..2dc1b8ca5
--- /dev/null
+++
b/seatunnel-engine/seatunnel-engine-client/src/test/resources/log4j2-test.properties
@@ -0,0 +1,42 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+rootLogger.level = INFO
+
+rootLogger.appenderRef.consoleStdout.ref = consoleStdoutAppender
+rootLogger.appenderRef.consoleStderr.ref = consoleStderrAppender
+
+appender.consoleStdout.name = consoleStdoutAppender
+appender.consoleStdout.type = CONSOLE
+appender.consoleStdout.target = SYSTEM_OUT
+appender.consoleStdout.layout.type = PatternLayout
+appender.consoleStdout.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %c -
%m%n
+appender.consoleStdout.filter.acceptLtWarn.type = ThresholdFilter
+appender.consoleStdout.filter.acceptLtWarn.level = WARN
+appender.consoleStdout.filter.acceptLtWarn.onMatch = DENY
+appender.consoleStdout.filter.acceptLtWarn.onMismatch = ACCEPT
+
+appender.consoleStderr.name = consoleStderrAppender
+appender.consoleStderr.type = CONSOLE
+appender.consoleStderr.target = SYSTEM_ERR
+appender.consoleStderr.layout.type = PatternLayout
+appender.consoleStderr.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %c -
%m%n
+appender.consoleStderr.filter.acceptGteWarn.type = ThresholdFilter
+appender.consoleStderr.filter.acceptGteWarn.level = WARN
+appender.consoleStderr.filter.acceptGteWarn.onMatch = ACCEPT
+appender.consoleStderr.filter.acceptGteWarn.onMismatch = DENY
diff --git
a/seatunnel-engine/seatunnel-engine-common/src/main/resources/hazelcast-client.yaml
b/seatunnel-engine/seatunnel-engine-common/src/main/resources/hazelcast-client.yaml
index 9552c382e..1a31be9f6 100644
---
a/seatunnel-engine/seatunnel-engine-common/src/main/resources/hazelcast-client.yaml
+++
b/seatunnel-engine/seatunnel-engine-common/src/main/resources/hazelcast-client.yaml
@@ -17,7 +17,8 @@
hazelcast-client:
cluster-name: seatunnel
-
+ properties:
+ hazelcast.logging.type: log4j2
network:
cluster-members:
- localhost:5801
diff --git
a/seatunnel-engine/seatunnel-engine-common/src/main/resources/hazelcast.yaml
b/seatunnel-engine/seatunnel-engine-common/src/main/resources/hazelcast.yaml
index 5ad9e5bc5..49c9bc20c 100644
--- a/seatunnel-engine/seatunnel-engine-common/src/main/resources/hazelcast.yaml
+++ b/seatunnel-engine/seatunnel-engine-common/src/main/resources/hazelcast.yaml
@@ -18,6 +18,11 @@
hazelcast:
cluster-name: seatunnel
network:
+ rest-api:
+ enabled: true
+ endpoint-groups:
+ CLUSTER_WRITE:
+ enabled: true
join:
tcp-ip:
enabled: true
@@ -30,3 +35,4 @@ hazelcast:
properties:
hazelcast.invocation.max.retry.count: 20
hazelcast.tcp.join.port.try.count: 30
+ hazelcast.logging.type: log4j2
diff --git
a/seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/NodeExtension.java
b/seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/NodeExtension.java
index e857d9bd8..f132a26c8 100644
---
a/seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/NodeExtension.java
+++
b/seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/NodeExtension.java
@@ -17,11 +17,18 @@
package org.apache.seatunnel.engine.server;
+import static
com.hazelcast.internal.ascii.TextCommandConstants.TextCommandType.HTTP_GET;
+import static
com.hazelcast.internal.ascii.TextCommandConstants.TextCommandType.HTTP_POST;
+
import org.apache.seatunnel.engine.common.config.SeaTunnelConfig;
+import org.apache.seatunnel.engine.server.log.Log4j2HttpGetCommandProcessor;
+import org.apache.seatunnel.engine.server.log.Log4j2HttpPostCommandProcessor;
import com.hazelcast.cluster.ClusterState;
import com.hazelcast.instance.impl.DefaultNodeExtension;
import com.hazelcast.instance.impl.Node;
+import com.hazelcast.internal.ascii.TextCommandService;
+import com.hazelcast.internal.ascii.TextCommandServiceImpl;
import lombok.NonNull;
import java.util.Map;
@@ -63,6 +70,16 @@ public class NodeExtension extends DefaultNodeExtension {
return extCommon.createExtensionServices();
}
+ @Override
+ public TextCommandService createTextCommandService() {
+ return new TextCommandServiceImpl(node) {
+ {
+ register(HTTP_GET, new Log4j2HttpGetCommandProcessor(this));
+ register(HTTP_POST, new Log4j2HttpPostCommandProcessor(this));
+ }
+ };
+ }
+
@Override
public void printNodeInfo() {
extCommon.printNodeInfo(systemLogger, "");
diff --git
a/seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/log/Log4j2HttpGetCommandProcessor.java
b/seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/log/Log4j2HttpGetCommandProcessor.java
new file mode 100644
index 000000000..604cc8570
--- /dev/null
+++
b/seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/log/Log4j2HttpGetCommandProcessor.java
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.engine.server.log;
+
+import com.hazelcast.internal.ascii.TextCommandService;
+import com.hazelcast.internal.ascii.rest.HttpCommandProcessor;
+import com.hazelcast.internal.ascii.rest.HttpGetCommand;
+import com.hazelcast.internal.ascii.rest.HttpGetCommandProcessor;
+import com.hazelcast.internal.json.JsonObject;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.core.LoggerContext;
+import org.apache.logging.log4j.core.config.LoggerConfig;
+
+import java.util.Map;
+
+public class Log4j2HttpGetCommandProcessor extends
HttpCommandProcessor<HttpGetCommand> {
+
+ private final HttpGetCommandProcessor original;
+
+ public Log4j2HttpGetCommandProcessor(TextCommandService
textCommandService) {
+ this(textCommandService, new
HttpGetCommandProcessor(textCommandService));
+ }
+
+ public Log4j2HttpGetCommandProcessor(TextCommandService textCommandService,
+ HttpGetCommandProcessor
httpGetCommandProcessor) {
+ super(textCommandService,
textCommandService.getNode().getLogger(Log4j2HttpGetCommandProcessor.class));
+ this.original = httpGetCommandProcessor;
+ }
+
+ @Override
+ public void handleRejection(HttpGetCommand request) {
+ handle(request);
+ }
+
+ @Override
+ public void handle(HttpGetCommand request) {
+ String uri = request.getURI();
+ if (uri.startsWith(HttpCommandProcessor.URI_LOG_LEVEL)) {
+ outputAllLoggerLevel(request);
+ } else {
+ original.handle(request);
+ }
+ }
+
+ /**
+ * Request example:
+ *
+ * GET {@link HttpCommandProcessor#URI_LOG_LEVEL}
+ *
+ * Response Body(application/json):
+ *
+ * {
+ * "root": "INFO"
+ * "com.example.logger1": "ERROR"
+ * }
+ *
+ */
+ private void outputAllLoggerLevel(HttpGetCommand request) {
+ JsonObject jsonObject = new JsonObject();
+
+ LoggerContext loggerContext = LoggerContext.getContext(false);
+ Map<String, LoggerConfig> loggers =
loggerContext.getConfiguration().getLoggers();
+ for (String logger : loggers.keySet()) {
+ LoggerConfig config = loggers.get(logger);
+ if (LogManager.ROOT_LOGGER_NAME.equals(logger)) {
+ logger = LoggerConfig.ROOT;
+ }
+ jsonObject.set(logger, config.getLevel().name());
+ }
+
+ prepareResponse(request, jsonObject);
+ textCommandService.sendResponse(request);
+ }
+}
diff --git
a/seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/log/Log4j2HttpPostCommandProcessor.java
b/seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/log/Log4j2HttpPostCommandProcessor.java
new file mode 100644
index 000000000..306d310ad
--- /dev/null
+++
b/seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/log/Log4j2HttpPostCommandProcessor.java
@@ -0,0 +1,90 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.engine.server.log;
+
+import static com.hazelcast.internal.ascii.rest.HttpStatusCode.SC_500;
+
+import com.hazelcast.internal.ascii.TextCommandService;
+import com.hazelcast.internal.ascii.rest.HttpCommandProcessor;
+import com.hazelcast.internal.ascii.rest.HttpPostCommand;
+import com.hazelcast.internal.ascii.rest.HttpPostCommandProcessor;
+import com.hazelcast.internal.json.JsonObject;
+import org.apache.logging.log4j.Level;
+import org.apache.logging.log4j.core.config.Configurator;
+import org.apache.logging.log4j.core.config.LoggerConfig;
+
+public class Log4j2HttpPostCommandProcessor extends
HttpCommandProcessor<HttpPostCommand> {
+
+ private final HttpPostCommandProcessor original;
+
+ public Log4j2HttpPostCommandProcessor(TextCommandService
textCommandService) {
+ this(textCommandService, new
HttpPostCommandProcessor(textCommandService));
+ }
+
+ public Log4j2HttpPostCommandProcessor(TextCommandService
textCommandService,
+ HttpPostCommandProcessor
httpPostCommandProcessor) {
+ super(textCommandService,
textCommandService.getNode().getLogger(Log4j2HttpPostCommandProcessor.class));
+ this.original = httpPostCommandProcessor;
+ }
+
+ @Override
+ public void handleRejection(HttpPostCommand request) {
+ handle(request);
+ }
+
+ @Override
+ public void handle(HttpPostCommand request) {
+ String uri = request.getURI();
+ if (uri.startsWith(HttpCommandProcessor.URI_LOG_LEVEL)) {
+ setLoggerLevel(request);
+ } else if (uri.startsWith(HttpCommandProcessor.URI_LOG_LEVEL_RESET)) {
+ prepareResponse(SC_500, request, "Reset logger level endpoint
disabled!");
+ textCommandService.sendResponse(request);
+ } else {
+ original.handle(request);
+ }
+ }
+
+ /**
+ * Request example:
+ *
+ * POST {@link HttpCommandProcessor#URI_LOG_LEVEL}
+ *
+ * Request Body(application/text):
+ *
+ * your_username&your_password&com.example.logger1&ERROR
+ *
+ */
+ @SuppressWarnings("MagicNumber")
+ private void setLoggerLevel(HttpPostCommand request) {
+ try {
+ String[] params = decodeParamsAndAuthenticate(request, 4);
+ String logger = params[2];
+ String level = params[3];
+ if (LoggerConfig.ROOT.equals(logger)) {
+ Configurator.setRootLevel(Level.getLevel(level));
+ } else {
+ Configurator.setLevel(logger, Level.getLevel(level));
+ }
+ prepareResponse(request, new JsonObject().add("status",
"SUCCESS"));
+ } catch (Throwable e) {
+ prepareResponse(SC_500, request, exceptionResponse(e));
+ }
+ textCommandService.sendResponse(request);
+ }
+}
diff --git
a/seatunnel-engine/seatunnel-engine-server/src/test/resources/log4j2-test.properties
b/seatunnel-engine/seatunnel-engine-server/src/test/resources/log4j2-test.properties
new file mode 100644
index 000000000..2dc1b8ca5
--- /dev/null
+++
b/seatunnel-engine/seatunnel-engine-server/src/test/resources/log4j2-test.properties
@@ -0,0 +1,42 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+rootLogger.level = INFO
+
+rootLogger.appenderRef.consoleStdout.ref = consoleStdoutAppender
+rootLogger.appenderRef.consoleStderr.ref = consoleStderrAppender
+
+appender.consoleStdout.name = consoleStdoutAppender
+appender.consoleStdout.type = CONSOLE
+appender.consoleStdout.target = SYSTEM_OUT
+appender.consoleStdout.layout.type = PatternLayout
+appender.consoleStdout.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %c -
%m%n
+appender.consoleStdout.filter.acceptLtWarn.type = ThresholdFilter
+appender.consoleStdout.filter.acceptLtWarn.level = WARN
+appender.consoleStdout.filter.acceptLtWarn.onMatch = DENY
+appender.consoleStdout.filter.acceptLtWarn.onMismatch = ACCEPT
+
+appender.consoleStderr.name = consoleStderrAppender
+appender.consoleStderr.type = CONSOLE
+appender.consoleStderr.target = SYSTEM_ERR
+appender.consoleStderr.layout.type = PatternLayout
+appender.consoleStderr.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %c -
%m%n
+appender.consoleStderr.filter.acceptGteWarn.type = ThresholdFilter
+appender.consoleStderr.filter.acceptGteWarn.level = WARN
+appender.consoleStderr.filter.acceptGteWarn.onMatch = ACCEPT
+appender.consoleStderr.filter.acceptGteWarn.onMismatch = DENY
diff --git
a/seatunnel-engine/seatunnel-engine-storage/checkpoint-storage-plugins/checkpoint-storage-local-file/src/test/resources/log4j2-test.properties
b/seatunnel-engine/seatunnel-engine-storage/checkpoint-storage-plugins/checkpoint-storage-local-file/src/test/resources/log4j2-test.properties
new file mode 100644
index 000000000..2dc1b8ca5
--- /dev/null
+++
b/seatunnel-engine/seatunnel-engine-storage/checkpoint-storage-plugins/checkpoint-storage-local-file/src/test/resources/log4j2-test.properties
@@ -0,0 +1,42 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+rootLogger.level = INFO
+
+rootLogger.appenderRef.consoleStdout.ref = consoleStdoutAppender
+rootLogger.appenderRef.consoleStderr.ref = consoleStderrAppender
+
+appender.consoleStdout.name = consoleStdoutAppender
+appender.consoleStdout.type = CONSOLE
+appender.consoleStdout.target = SYSTEM_OUT
+appender.consoleStdout.layout.type = PatternLayout
+appender.consoleStdout.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %c -
%m%n
+appender.consoleStdout.filter.acceptLtWarn.type = ThresholdFilter
+appender.consoleStdout.filter.acceptLtWarn.level = WARN
+appender.consoleStdout.filter.acceptLtWarn.onMatch = DENY
+appender.consoleStdout.filter.acceptLtWarn.onMismatch = ACCEPT
+
+appender.consoleStderr.name = consoleStderrAppender
+appender.consoleStderr.type = CONSOLE
+appender.consoleStderr.target = SYSTEM_ERR
+appender.consoleStderr.layout.type = PatternLayout
+appender.consoleStderr.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %c -
%m%n
+appender.consoleStderr.filter.acceptGteWarn.type = ThresholdFilter
+appender.consoleStderr.filter.acceptGteWarn.level = WARN
+appender.consoleStderr.filter.acceptGteWarn.onMatch = ACCEPT
+appender.consoleStderr.filter.acceptGteWarn.onMismatch = DENY
diff --git a/tools/dependencies/known-dependencies.txt
b/tools/dependencies/known-dependencies.txt
index db0b6fbaa..a4b2de5ec 100755
--- a/tools/dependencies/known-dependencies.txt
+++ b/tools/dependencies/known-dependencies.txt
@@ -18,6 +18,7 @@ commons-lang3-3.4.jar
commons-math3-3.5.jar
commons-pool-1.6.jar
config-1.3.3.jar
+disruptor-3.4.4.jar
flink-annotations-1.13.6.jar
flink-avro-1.13.6.jar
flink-connector-base-1.13.6.jar
@@ -60,10 +61,14 @@ jackson-dataformat-properties-2.12.6.jar
jackson-mapper-asl-1.9.13.jar
javassist-3.24.0-GA.jar
javax.annotation-api-1.3.2.jar
+jcl-over-slf4j-1.7.25.jar
jcommander-1.81.jar
jsr305-1.3.9.jar
kryo-2.24.0.jar
-log4j-1.2.17.jar
+log4j-api-2.17.1.jar
+log4j-core-2.17.1.jar
+log4j-slf4j-impl-2.17.1.jar
+log4j-1.2-api-2.17.1.jar
lz4-1.3.0.jar
lz4-java-1.6.0.jar
minlog-1.2.jar
@@ -100,7 +105,6 @@ scopt_2.11-3.5.0.jar
seatunnel-config-base-2.1.1.jar
seatunnel-config-shade-2.1.1.jar
slf4j-api-1.7.25.jar
-slf4j-log4j12-1.7.25.jar
snappy-java-1.1.1.3.jar
snappy-java-1.1.7.1.jar
snappy-java-1.1.8.3.jar
