[
https://issues.apache.org/jira/browse/FLINK-18284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134841#comment-17134841
]
appleyuchi commented on FLINK-18284:
------------------------------------
*修复效果如下:*
(Python3.6) appleyuchi@Desktop:~$ start-scala-shell.sh local
Starting Flink Shell:
13/06/2020 23:23:51.327 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: jobmanager.rpc.address, Desktop
13/06/2020 23:23:51.327 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: jobmanager.rpc.port, 6123
13/06/2020 23:23:51.327 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: jobmanager.heap.size, 2048m
13/06/2020 23:23:51.327 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: taskmanager.memory.process.size, 1728m
13/06/2020 23:23:51.328 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: taskmanager.numberOfTaskSlots, 2
13/06/2020 23:23:51.328 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: parallelism.default, 2
13/06/2020 23:23:51.328 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: jobmanager.execution.failover-strategy, region
13/06/2020 23:23:51.328 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: rest.port, 8082
13/06/2020 23:23:51.329 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: rest.address, Desktop
13/06/2020 23:23:51.329 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: web.submit.enable, true
13/06/2020 23:23:51.329 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: jobmanager.archive.fs.dir, hdfs://Desktop:9000/completed-jobs/
13/06/2020 23:23:51.329 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: historyserver.web.address, Desktop
13/06/2020 23:23:51.329 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: historyserver.web.port, 18082
13/06/2020 23:23:51.330 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: historyserver.archive.fs.dir, hdfs://Desktop:9000/completed-jobs/
13/06/2020 23:23:51.330 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: historyserver.archive.fs.refresh-interval, 10000
13/06/2020 23:23:51.330 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: env.java.home, /home/appleyuchi/Java/jdk1.8.0_131
13/06/2020 23:23:51.400 WARN
[org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils] The resource
configuration option Key: 'taskmanager.memory.process.size' , default: null
(fallback keys: []) is set but it will have no effect for local execution, only
the following options matter for the resource configuration: [Key:
'taskmanager.memory.process.size' , default: null (fallback keys: []), Key:
'taskmanager.memory.flink.size' , default: null (fallback keys: []), Key:
'taskmanager.memory.framework.heap.size' , default: 128 mb (fallback keys: []),
Key: 'taskmanager.memory.framework.off-heap.size' , default: 128 mb (fallback
keys: []), Key: 'taskmanager.memory.jvm-metaspace.size' , default: 256 mb
(fallback keys: []), Key: 'taskmanager.memory.jvm-overhead.min' , default: 192
mb (fallback keys: []), Key: 'taskmanager.memory.jvm-overhead.max' , default: 1
gb (fallback keys: []), Key: 'taskmanager.memory.jvm-overhead.fraction' ,
default: 0.1 (fallback keys: [])]
13/06/2020 23:23:51.400 INFO
[org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils] The
configuration option Key: 'taskmanager.cpu.cores' , default: null (fallback
keys: []) required for local execution is not set, setting it to its default
value 1.7976931348623157E308
13/06/2020 23:23:51.400 INFO
[org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils] The
configuration option Key: 'taskmanager.memory.task.heap.size' , default: null
(fallback keys: []) required for local execution is not set, setting it to its
default value 9223372036854775807 bytes
13/06/2020 23:23:51.400 INFO
[org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils] The
configuration option Key: 'taskmanager.memory.task.off-heap.size' , default: 0
bytes (fallback keys: []) required for local execution is not set, setting it
to its default value 9223372036854775807 bytes
13/06/2020 23:23:51.401 INFO
[org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils] The
configuration option Key: 'taskmanager.memory.network.min' , default: 64 mb
(fallback keys: [\{key=taskmanager.network.memory.min, isDeprecated=true}])
required for local execution is not set, setting it to its default value 64 mb
13/06/2020 23:23:51.401 INFO
[org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils] The
configuration option Key: 'taskmanager.memory.network.max' , default: 1 gb
(fallback keys: [\{key=taskmanager.network.memory.max, isDeprecated=true}])
required for local execution is not set, setting it to its default value 64 mb
13/06/2020 23:23:51.401 INFO
[org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils] The
configuration option Key: 'taskmanager.memory.managed.size' , default: null
(fallback keys: [\{key=taskmanager.memory.size, isDeprecated=true}]) required
for local execution is not set, setting it to its default value 128 mb
13/06/2020 23:23:51.414 INFO [org.apache.flink.runtime.minicluster.MiniCluster]
Starting Flink Mini Cluster
13/06/2020 23:23:51.416 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: jobmanager.rpc.address, Desktop
13/06/2020 23:23:51.416 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: jobmanager.rpc.port, 6123
13/06/2020 23:23:51.416 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: jobmanager.heap.size, 2048m
13/06/2020 23:23:51.416 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: taskmanager.memory.process.size, 1728m
13/06/2020 23:23:51.416 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: taskmanager.numberOfTaskSlots, 2
13/06/2020 23:23:51.416 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: parallelism.default, 2
13/06/2020 23:23:51.416 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: jobmanager.execution.failover-strategy, region
13/06/2020 23:23:51.416 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: rest.port, 8082
13/06/2020 23:23:51.416 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: rest.address, Desktop
13/06/2020 23:23:51.417 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: web.submit.enable, true
13/06/2020 23:23:51.417 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: jobmanager.archive.fs.dir, hdfs://Desktop:9000/completed-jobs/
13/06/2020 23:23:51.417 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: historyserver.web.address, Desktop
13/06/2020 23:23:51.417 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: historyserver.web.port, 18082
13/06/2020 23:23:51.417 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: historyserver.archive.fs.dir, hdfs://Desktop:9000/completed-jobs/
13/06/2020 23:23:51.417 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: historyserver.archive.fs.refresh-interval, 10000
13/06/2020 23:23:51.417 INFO
[org.apache.flink.configuration.GlobalConfiguration] Loading configuration
property: env.java.home, /home/appleyuchi/Java/jdk1.8.0_131
13/06/2020 23:23:51.419 INFO [org.apache.flink.runtime.minicluster.MiniCluster]
Starting Metrics Registry
13/06/2020 23:23:51.460 INFO
[org.apache.flink.runtime.metrics.MetricRegistryImpl] No metrics reporter
configured, no metrics will be exposed/reported.
13/06/2020 23:23:51.460 INFO [org.apache.flink.runtime.minicluster.MiniCluster]
Starting RPC Service(s)
13/06/2020 23:23:51.731 INFO [akka.event.slf4j.Slf4jLogger] Slf4jLogger started
13/06/2020 23:23:51.799 INFO
[org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils] Trying to start actor
system at :0
13/06/2020 23:23:51.838 INFO [akka.event.slf4j.Slf4jLogger] Slf4jLogger started
13/06/2020 23:23:51.851 INFO [akka.remote.Remoting] Starting remoting
13/06/2020 23:23:51.958 INFO [akka.remote.Remoting] Remoting started; listening
on addresses :[akka.tcp://[email protected]:36023]
13/06/2020 23:23:51.967 INFO
[org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils] Actor system started at
akka.tcp://[email protected]:36023
13/06/2020 23:23:51.972 INFO [org.apache.flink.runtime.rpc.akka.AkkaRpcService]
Starting RPC endpoint for
org.apache.flink.runtime.metrics.dump.MetricQueryService at
akka://flink-metrics/user/MetricQueryService .
13/06/2020 23:23:52.058 INFO [org.apache.flink.runtime.minicluster.MiniCluster]
Starting high-availability services
13/06/2020 23:23:52.071 INFO [org.apache.flink.runtime.blob.BlobServer] Created
BLOB server storage directory
/tmp/blobStore-2eb23fd1-6af2-4ef4-9990-c7026b398cdc
13/06/2020 23:23:52.075 INFO [org.apache.flink.runtime.blob.BlobServer] Started
BLOB server at 0.0.0.0:43421 - max concurrent requests: 50 - max backlog: 1000
13/06/2020 23:23:52.078 INFO [org.apache.flink.runtime.blob.PermanentBlobCache]
Created BLOB cache storage directory
/tmp/blobStore-e2d31c1c-ef19-4fd9-9f79-f0317a327f01
13/06/2020 23:23:52.079 INFO [org.apache.flink.runtime.blob.TransientBlobCache]
Created BLOB cache storage directory
/tmp/blobStore-00bc8023-74d0-4f3d-9a74-dd31aaf49177
13/06/2020 23:23:52.080 INFO [org.apache.flink.runtime.minicluster.MiniCluster]
Starting 1 TaskManger(s)
13/06/2020 23:23:52.081 INFO
[org.apache.flink.runtime.taskexecutor.TaskManagerRunner] Starting TaskManager
with ResourceID: 3cf127c5-8005-4b60-b728-7e52ca0293a5
13/06/2020 23:23:52.088 INFO
[org.apache.flink.runtime.taskexecutor.TaskManagerServices] Temporary file
directory '/tmp': total 136 GB, usable 61 GB (44.85% usable)
13/06/2020 23:23:52.090 INFO
[org.apache.flink.runtime.io.disk.FileChannelManagerImpl] FileChannelManager
uses directory /tmp/flink-io-48f2f98e-0fb4-44f6-92e3-f37b58f7fd05 for spill
files.
13/06/2020 23:23:52.097 INFO
[org.apache.flink.runtime.io.disk.FileChannelManagerImpl] FileChannelManager
uses directory /tmp/flink-netty-shuffle-0dcc5d28-4052-4486-b596-a70702c14cee
for spill files.
13/06/2020 23:23:52.131 INFO
[org.apache.flink.runtime.io.network.buffer.NetworkBufferPool] Allocated 64 MB
for network buffer pool (number of memory segments: 2048, bytes per segment:
32768).
13/06/2020 23:23:52.136 INFO
[org.apache.flink.runtime.io.network.NettyShuffleEnvironment] Starting the
network environment and its components.
13/06/2020 23:23:52.137 INFO
[org.apache.flink.runtime.taskexecutor.KvStateService] Starting the kvState
service and its components.
13/06/2020 23:23:52.143 INFO
[org.apache.flink.runtime.taskexecutor.TaskManagerConfiguration] Messages have
a max timeout of 10000 ms
13/06/2020 23:23:52.166 INFO [org.apache.flink.runtime.rpc.akka.AkkaRpcService]
Starting RPC endpoint for org.apache.flink.runtime.taskexecutor.TaskExecutor at
akka://flink/user/taskmanager_0 .
13/06/2020 23:23:52.175 INFO
[org.apache.flink.runtime.taskexecutor.JobLeaderService] Start job leader
service.
13/06/2020 23:23:52.176 INFO [org.apache.flink.runtime.filecache.FileCache]
User file cache uses directory
/tmp/flink-dist-cache-4a2ebd2f-6749-4a68-9807-9b8b00f7815a
13/06/2020 23:23:52.188 INFO [org.apache.flink.configuration.Configuration]
Config uses fallback configuration key 'rest.port' instead of key
'rest.bind-port'
13/06/2020 23:23:52.215 INFO
[org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint] Starting rest
endpoint.
13/06/2020 23:23:52.376 INFO
[org.apache.flink.runtime.webmonitor.WebMonitorUtils] Determined location of
main cluster component log file:
/home/appleyuchi/bigdata/flink-1.10.1/log/flink-appleyuchi-scala-shell-local-Desktop.log
13/06/2020 23:23:52.376 INFO
[org.apache.flink.runtime.webmonitor.WebMonitorUtils] Determined location of
main cluster component stdout file:
/home/appleyuchi/bigdata/flink-1.10.1/log/flink-appleyuchi-scala-shell-local-Desktop.out
13/06/2020 23:23:52.492 INFO
[org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint] Rest endpoint
listening at Desktop:8082
13/06/2020 23:23:52.493 INFO
[org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService]
Proposing leadership to contender http://Desktop:8082
13/06/2020 23:23:52.495 INFO
[org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint] Web frontend
listening at http://Desktop:8082.
13/06/2020 23:23:52.496 INFO
[org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint]
http://Desktop:8082 was granted leadership with
leaderSessionID=7d7abb46-cfe8-4e9b-b040-061c57fede32
13/06/2020 23:23:52.496 INFO
[org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService]
Received confirmation of leadership for leader http://Desktop:8082 ,
session=7d7abb46-cfe8-4e9b-b040-061c57fede32
13/06/2020 23:23:52.503 INFO [org.apache.flink.runtime.rpc.akka.AkkaRpcService]
Starting RPC endpoint for
org.apache.flink.runtime.resourcemanager.StandaloneResourceManager at
akka://flink/user/resourcemanager .
13/06/2020 23:23:52.522 INFO
[org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService]
Proposing leadership to contender LeaderContender: DefaultDispatcherRunner
13/06/2020 23:23:52.523 INFO
[org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService]
Proposing leadership to contender LeaderContender: StandaloneResourceManager
13/06/2020 23:23:52.525 INFO
[org.apache.flink.runtime.resourcemanager.StandaloneResourceManager]
ResourceManager akka://flink/user/resourcemanager was granted leadership with
fencing token b39e28df051dee22e5aaabc9a22345b7
13/06/2020 23:23:52.526 INFO [org.apache.flink.runtime.minicluster.MiniCluster]
Flink Mini Cluster started successfully
13/06/2020 23:23:52.528 INFO
[org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl] Starting
the SlotManager.
Starting local Flink cluster (host: localhost, port: 8082).
13/06/2020 23:23:52.530 INFO
[org.apache.flink.runtime.dispatcher.runner.SessionDispatcherLeaderProcess]
Start SessionDispatcherLeaderProcess.
13/06/2020 23:23:52.530 INFO
[org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService]
Received confirmation of leadership for leader
akka://flink/user/resourcemanager , session=e5aaabc9-a223-45b7-b39e-28df051dee22
Connecting to Flink cluster (host: localhost, port: 8082).
13/06/2020 23:23:52.533 INFO
[org.apache.flink.runtime.dispatcher.runner.SessionDispatcherLeaderProcess]
Recover all persisted job graphs.
13/06/2020 23:23:52.533 INFO
[org.apache.flink.runtime.dispatcher.runner.SessionDispatcherLeaderProcess]
Successfully recovered 0 persisted job graphs.
13/06/2020 23:23:52.539 INFO [org.apache.flink.runtime.rpc.akka.AkkaRpcService]
Starting RPC endpoint for
org.apache.flink.runtime.dispatcher.StandaloneDispatcher at
akka://flink/user/dispatcher .
13/06/2020 23:23:52.542 INFO
[org.apache.flink.runtime.taskexecutor.TaskExecutor] Connecting to
ResourceManager
akka://flink/user/resourcemanager(b39e28df051dee22e5aaabc9a22345b7).
13/06/2020 23:23:52.557 INFO
[org.apache.flink.runtime.taskexecutor.TaskExecutor] Resolved ResourceManager
address, beginning registration
13/06/2020 23:23:52.557 INFO
[org.apache.flink.runtime.taskexecutor.TaskExecutor] Registration at
ResourceManager attempt 1 (timeout=100ms)
13/06/2020 23:23:52.569 INFO
[org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService]
Received confirmation of leadership for leader akka://flink/user/dispatcher ,
session=9928799b-9ad4-4223-939b-da0599dbd704
13/06/2020 23:23:52.569 INFO
[org.apache.flink.runtime.resourcemanager.StandaloneResourceManager]
Registering TaskManager with ResourceID 3cf127c5-8005-4b60-b728-7e52ca0293a5
(akka://flink/user/taskmanager_0) at ResourceManager
13/06/2020 23:23:52.573 INFO
[org.apache.flink.runtime.taskexecutor.TaskExecutor] Successful registration at
resource manager akka://flink/user/resourcemanager under registration id
7d155a856666cd66d0b8a08edcf9514b.
▒▓██▓██▒
▓████▒▒█▓▒▓███▓▒
▓███▓░░ ▒▒▒▓██▒ ▒
░██▒ ▒▒▓▓█▓▓▒░ ▒████
██▒ ░▒▓███▒ ▒█▒█▒
░▓█ ███ ▓░▒██
▓█ ▒▒▒▒▒▓██▓░▒░▓▓█
█░ █ ▒▒░ ███▓▓█ ▒█▒▒▒
████░ ▒▓█▓ ██▒▒▒ ▓███▒
░▒█▓▓██ ▓█▒ ▓█▒▓██▓ ░█░
▓░▒▓████▒ ██ ▒█ █▓░▒█▒░▒█▒
███▓░██▓ ▓█ █ █▓ ▒▓█▓▓█▒
░██▓ ░█░ █ █▒ ▒█████▓▒ ██▓░▒
███░ ░ █░ ▓ ░█ █████▒░░ ░█░▓ ▓░
██▓█ ▒▒▓▒ ▓███████▓░ ▒█▒ ▒▓ ▓██▓
▒██▓ ▓█ █▓█ ░▒█████▓▓▒░ ██▒▒ █ ▒ ▓█▒
▓█▓ ▓█ ██▓ ░▓▓▓▓▓▓▓▒ ▒██▓ ░█▒
▓█ █ ▓███▓▒░ ░▓▓▓███▓ ░▒░ ▓█
██▓ ██▒ ░▒▓▓███▓▓▓▓▓██████▓▒ ▓███ █
▓███▒ ███ ░▓▓▒░░ ░▓████▓░ ░▒▓▒ █▓
█▓▒▒▓▓██ ░▒▒░░░▒▒▒▒▓██▓░ █▓
██ ▓░▒█ ▓▓▓▓▒░░ ▒█▓ ▒▓▓██▓ ▓▒ ▒▒▓
▓█▓ ▓▒█ █▓░ ░▒▓▓██▒ ░▓█▒ ▒▒▒░▒▒▓█████▒
██░ ▓█▒█▒ ▒▓▓▒ ▓█ █░ ░░░░ ░█▒
▓█ ▒█▓ ░ █░ ▒█ █▓
█▓ ██ █░ ▓▓ ▒█▓▓▓▒█░
█▓ ░▓██░ ▓▒ ▓█▓▒░░░▒▓█░ ▒█
██ ▓█▓░ ▒ ░▒█▒██▒ ▓▓
▓█▒ ▒█▓▒░ ▒▒ █▒█▓▒▒░░▒██
░██▒ ▒▓▓▒ ▓██▓▒█▒ ░▓▓▓▓▒█▓
░▓██▒ ▓░ ▒█▓█ ░░▒▒▒
▒▓▓▓▓▓▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒░░▓▓ ▓░▒█░
F L I N K - S C A L A - S H E L L
NOTE: Use the prebound Execution Environments and Table Environment to
implement batch or streaming programs.
Batch - Use the 'benv' and 'btenv' variable
* val dataSet = benv.readTextFile("/path/to/data")
* dataSet.writeAsText("/path/to/output")
* benv.execute("My batch program")
*
* val batchTable = btenv.fromDataSet(dataSet)
* btenv.registerTable("tableName", batchTable)
* val result = btenv.sqlQuery("SELECT * FROM tableName").collect
HINT: You can use print() on a DataSet to print the contents or collect()
a sql query result back to the shell.
Streaming - Use the 'senv' and 'stenv' variable
* val dataStream = senv.fromElements(1, 2, 3, 4)
* dataStream.countWindowAll(2).sum(0).print()
*
* val streamTable = stenv.fromDataStream(dataStream, 'num)
* val resultTable = streamTable.select('num).where('num % 2 === 1 )
* resultTable.toAppendStream[Row].print()
* senv.execute("My streaming program")
HINT: You can only print a DataStream to the shell in local mode.
scala>
> No appenders could be found for logger
> ---------------------------------------
>
> Key: FLINK-18284
> URL: https://issues.apache.org/jira/browse/FLINK-18284
> Project: Flink
> Issue Type: Bug
> Affects Versions: 1.11.1
> Reporter: appleyuchi
> Priority: Major
>
> start-scala-shell.sh local
> Starting Flink Shell:
> log4j:WARN No appenders could be found for logger
> (org.apache.flink.configuration.GlobalConfiguration).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more
> info.
>
> what's the configuration to fix this warning,it's annoying.
> PS:
> configuration in log4j.properties did not work.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
