leo65535 commented on pull request #885:
URL:
https://github.com/apache/incubator-seatunnel/pull/885#issuecomment-1002470990
hi @simon824, here is the lstest result
```
[dcadmin@dcadmin seatunnel-dist-2.0.5-SNAPSHOT-2.11.8]$ vi
config/application.conf
[dcadmin@dcadmin seatunnel-dist-2.0.5-SNAPSHOT-2.11.8]$
./bin/start-seatunnel-spark.sh -c config/application.conf -e client -m local -i
app=test -t
[INFO] spark conf: --conf "spark.app.name=seatunnel"
Warning: Ignoring non-spark config property: "spark.app.name=seatunnel"
2021-12-29 15:31:55 WARN NativeCodeLoader:62 - Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
2021-12-29 15:31:55 INFO ConfigBuilder:78 - Loading config file:
config/application.conf
2021-12-29 15:31:55 INFO ConfigBuilder:89 - parsed config file: {
"env" : {
"spark.app.name" : "seatunnel"
},
"source" : [
{
"result_table_name" : "my_dataset",
"plugin_name" : "Fake"
}
],
"transform" : [
{
"plugin_name" : "sql",
"table_name" : "user_view",
"sql" : ${?table_name}"select * from user_view where city =
'"${?sql}"'"
}
],
"sink" : [
{
"plugin_name" : "Console"
}
]
}
2021-12-29 15:31:55 INFO SparkContext:54 - Running Spark version 2.4.0
2021-12-29 15:31:55 INFO SparkContext:54 - Submitted application: seatunnel
2021-12-29 15:31:55 INFO SecurityManager:54 - Changing view acls to: dcadmin
2021-12-29 15:31:55 INFO SecurityManager:54 - Changing modify acls to:
dcadmin
2021-12-29 15:31:55 INFO SecurityManager:54 - Changing view acls groups to:
2021-12-29 15:31:55 INFO SecurityManager:54 - Changing modify acls groups
to:
2021-12-29 15:31:55 INFO SecurityManager:54 - SecurityManager:
authentication disabled; ui acls disabled; users with view permissions:
Set(dcadmin); groups with view permissions: Set(); users with modify
permissions: Set(dcadmin); groups with modify permissions: Set()
2021-12-29 15:31:56 INFO Utils:54 - Successfully started service
'sparkDriver' on port 44517.
2021-12-29 15:31:56 INFO SparkEnv:54 - Registering MapOutputTracker
2021-12-29 15:31:56 INFO SparkEnv:54 - Registering BlockManagerMaster
2021-12-29 15:31:56 INFO BlockManagerMasterEndpoint:54 - Using
org.apache.spark.storage.DefaultTopologyMapper for getting topology information
2021-12-29 15:31:56 INFO BlockManagerMasterEndpoint:54 -
BlockManagerMasterEndpoint up
2021-12-29 15:31:56 INFO DiskBlockManager:54 - Created local directory at
/tmp/blockmgr-9bc23ab1-2048-4316-a7ca-fd88799ab719
2021-12-29 15:31:56 INFO MemoryStore:54 - MemoryStore started with capacity
366.3 MB
2021-12-29 15:31:56 INFO SparkEnv:54 - Registering OutputCommitCoordinator
2021-12-29 15:31:56 INFO log:192 - Logging initialized @3512ms
2021-12-29 15:31:56 INFO Server:351 - jetty-9.3.z-SNAPSHOT, build
timestamp: unknown, git hash: unknown
2021-12-29 15:31:56 INFO Server:419 - Started @3615ms
2021-12-29 15:31:56 INFO AbstractConnector:278 - Started
ServerConnector@6079cf5{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}
2021-12-29 15:31:56 INFO Utils:54 - Successfully started service 'SparkUI'
on port 4040.
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@6edaa77a{/jobs,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@749f539e{/jobs/json,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@5ca1f591{/jobs/job,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@6ef81f31{/jobs/job/json,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@6075b2d3{/stages,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@33abde31{/stages/json,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@997d532{/stages/stage,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@7a18e8d{/stages/stage/json,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@3028e50e{/stages/pool,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@5560bcdf{/stages/pool/json,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@b558294{/storage,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@bb095{/storage/json,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@777c350f{/storage/rdd,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@27aae97b{/storage/rdd/json,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@4c9e38{/environment,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@5d1e09bc{/environment/json,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@4bdc8b5d{/executors,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@3bcd426c{/executors/json,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@5f14a673{/executors/threadDump,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@726a17c4{/executors/threadDump/json,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@5dc3fcb7{/static,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@6b3871d6{/,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@37095ded{/api,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@7923f5b3{/jobs/job/kill,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@6b63d445{/stages/stage/kill,null,AVAILABLE,@Spark}
2021-12-29 15:31:56 INFO SparkUI:54 - Bound SparkUI to 0.0.0.0, and started
at http://dcadmin.work:4040
2021-12-29 15:31:56 INFO SparkContext:54 - Added JAR
file:///work/projects/opensource/seatunnel/seatunnel-dist/seatunnel-dist-2.0.5-SNAPSHOT-2.11.8/lib/seatunnel-core-flink.jar
at spark://dcadmin.work:44517/jars/seatunnel-core-flink.jar with timestamp
1640763116838
2021-12-29 15:31:56 INFO SparkContext:54 - Added JAR
file:///work/projects/opensource/seatunnel/seatunnel-dist/seatunnel-dist-2.0.5-SNAPSHOT-2.11.8/lib/seatunnel-core-spark.jar
at spark://dcadmin.work:44517/jars/seatunnel-core-spark.jar with timestamp
1640763116839
2021-12-29 15:31:56 INFO SparkContext:54 - Added JAR
file:///work/projects/opensource/seatunnel/seatunnel-dist/seatunnel-dist-2.0.5-SNAPSHOT-2.11.8/lib/seatunnel-core-sql-2.0.5-SNAPSHOT-2.11.8.jar
at
spark://dcadmin.work:44517/jars/seatunnel-core-sql-2.0.5-SNAPSHOT-2.11.8.jar
with timestamp 1640763116840
2021-12-29 15:31:56 WARN SparkContext:66 - The jar
file:/work/projects/opensource/seatunnel/seatunnel-dist/seatunnel-dist-2.0.5-SNAPSHOT-2.11.8/lib/seatunnel-core-spark.jar
has been added already. Overwriting of added jars is not supported in the
current version.
2021-12-29 15:31:56 INFO Executor:54 - Starting executor ID driver on host
localhost
2021-12-29 15:31:57 INFO Utils:54 - Successfully started service
'org.apache.spark.network.netty.NettyBlockTransferService' on port 40581.
2021-12-29 15:31:57 INFO NettyBlockTransferService:54 - Server created on
dcadmin.work:40581
2021-12-29 15:31:57 INFO BlockManager:54 - Using
org.apache.spark.storage.RandomBlockReplicationPolicy for block replication
policy
2021-12-29 15:31:57 INFO BlockManagerMaster:54 - Registering BlockManager
BlockManagerId(driver, dcadmin.work, 40581, None)
2021-12-29 15:31:57 INFO BlockManagerMasterEndpoint:54 - Registering block
manager dcadmin.work:40581 with 366.3 MB RAM, BlockManagerId(driver,
dcadmin.work, 40581, None)
2021-12-29 15:31:57 INFO BlockManagerMaster:54 - Registered BlockManager
BlockManagerId(driver, dcadmin.work, 40581, None)
2021-12-29 15:31:57 INFO BlockManager:54 - Initialized BlockManager:
BlockManagerId(driver, dcadmin.work, 40581, None)
2021-12-29 15:31:57 INFO ContextHandler:781 - Started
o.s.j.s.ServletContextHandler@77a2aa4a{/metrics/json,null,AVAILABLE,@Spark}
2021-12-29 15:31:57 WARN StreamingContext:66 - spark.master should be set
as local[n], n > 1 in local mode if you have receivers to get data, otherwise
Spark jobs will not get resources to process the received data.
2021-12-29 15:31:57 ERROR Seatunnel:114 -
Plugin[org.apache.seatunnel.spark.transform.Sql] contains invalid config,
error: need to Config#resolve(), see the API docs for Config#resolve();
substitution not resolved: ConfigConcatenation(${?table_name}"select * from
user_view where city = '"${?sql}"'")
2021-12-29 15:31:57 INFO SparkContext:54 - Invoking stop() from shutdown
hook
2021-12-29 15:31:57 INFO AbstractConnector:318 - Stopped
Spark@6079cf5{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}
2021-12-29 15:31:57 INFO SparkUI:54 - Stopped Spark web UI at
http://dcadmin.work:4040
2021-12-29 15:31:57 INFO MapOutputTrackerMasterEndpoint:54 -
MapOutputTrackerMasterEndpoint stopped!
2021-12-29 15:31:57 INFO MemoryStore:54 - MemoryStore cleared
2021-12-29 15:31:57 INFO BlockManager:54 - BlockManager stopped
2021-12-29 15:31:57 INFO BlockManagerMaster:54 - BlockManagerMaster stopped
2021-12-29 15:31:57 INFO
OutputCommitCoordinator$OutputCommitCoordinatorEndpoint:54 -
OutputCommitCoordinator stopped!
2021-12-29 15:31:57 INFO SparkContext:54 - Successfully stopped SparkContext
2021-12-29 15:31:57 INFO ShutdownHookManager:54 - Shutdown hook called
2021-12-29 15:31:57 INFO ShutdownHookManager:54 - Deleting directory
/tmp/spark-3f3ab240-66f9-4ab3-baad-050542b74e86
2021-12-29 15:31:57 INFO ShutdownHookManager:54 - Deleting directory
/tmp/spark-4231815e-59e3-49fe-a258-7d41780e030f
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
