[
https://issues.apache.org/jira/browse/FLINK-17977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17124933#comment-17124933
]
Chesnay Schepler edited comment on FLINK-17977 at 6/3/20, 1:07 PM:
-------------------------------------------------------------------
The following warning shows up if hadoop is not available. It shouldn't be a
warning, and it is not clear what is meant with "incompatible":
{code}
2020-06-03 14:44:45,520 WARN org.apache.flink.runtime.security.SecurityUtils
[] - Unable to install incompatible security context factory
org.apache.flink.runtime.security.contexts.HadoopSecurityContextFactory
{code}
This one occurring during task executor startup is a bit unclear:
{code}
2020-06-03 14:50:39,367 INFO org.apache.flink.runtime.net.ConnectionUtils
[] - Retrieved new target address localhost/127.0.0.1:6123.
{code}
These could probably be debug messages:
{code}
2020-06-03 14:50:41,571 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerConfiguration [] - Messages
have a max timeout of 10000 ms
...
2020-06-03 14:50:41,945 INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Registration
at ResourceManager attempt 1 (timeout=100ms)
2020-06-03 14:50:42,093 INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Registration
at ResourceManager attempt 2 (timeout=200ms)
{code}
When the first TaskManager registration times out you get this INFO message,
which somewhat comes out of nowhere and doesn't give much valuable information;
could probably be DEBUG instead:
{code}
020-06-03 14:50:42,109 INFO
org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] -
Ignoring outdated TaskExecutorGateway connection.
{code}
Running a simple streaming wordcount resulted in this smelly message:
{code}
2020-06-03 14:58:33,870 INFO
org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl [] -
JobManagerRunner already shutdown.
{code}
These filesystem messages are really noisy:
{code}
2020-06-03 14:58:32,994 INFO org.apache.flink.runtime.taskmanager.Task
[] - Creating FileSystem stream leak safety net for task Keyed
Aggregation -> Sink: Print to Std. Out (1/1) (7f2$
2020-06-03 14:58:33,662 INFO org.apache.flink.runtime.taskmanager.Task
[] - Ensuring all FileSystem streams are closed for task Source:
Collection Source -> Flat Map (1/1) (12bf05d6f3$
{code}
Table programs log several repeated messages:
{code}
2020-06-03 15:02:11,982 INFO org.apache.flink.table.module.ModuleManager
[] - Got FunctionDefinition 'as' from 'core' module.
2020-06-03 15:02:11,988 INFO org.apache.flink.table.module.ModuleManager
[] - Got FunctionDefinition 'sum' from 'core' module.
2020-06-03 15:02:12,139 INFO org.apache.flink.table.module.ModuleManager
[] - Got FunctionDefinition 'as' from 'core' module.
2020-06-03 15:02:12,159 INFO org.apache.flink.table.module.ModuleManager
[] - Got FunctionDefinition 'equals' from 'core' module.
2020-06-03 15:02:14,932 INFO org.apache.flink.api.java.typeutils.TypeExtractor
[] - class org.apache.flink.types.Row does not contain a getter for
field fields
2020-06-03 15:02:14,932 INFO org.apache.flink.api.java.typeutils.TypeExtractor
[] - class org.apache.flink.types.Row does not contain a setter for
field fields
2020-06-03 15:02:14,932 INFO org.apache.flink.api.java.typeutils.TypeExtractor
[] - Class class org.apache.flink.types.Row cannot be used as a POJO
type because not all fields are valid POJO $
2020-06-03 15:02:15,073 INFO org.apache.flink.api.java.typeutils.TypeExtractor
[] - class org.apache.flink.types.Row does not contain a getter for
field fields
2020-06-03 15:02:15,073 INFO org.apache.flink.api.java.typeutils.TypeExtractor
[] - class org.apache.flink.types.Row does not contain a setter for
field fields
2020-06-03 15:02:15,073 INFO org.apache.flink.api.java.typeutils.TypeExtractor
[] - Class class org.apache.flink.types.Row cannot be used as a POJO
type because not all fields are valid POJO $
2020-06-03 15:02:15,098 INFO org.apache.flink.api.java.typeutils.TypeExtractor
[] - class org.apache.flink.types.Row does not contain a getter for
field fields
2020-06-03 15:02:15,098 INFO org.apache.flink.api.java.typeutils.TypeExtractor
[] - class org.apache.flink.types.Row does not contain a setter for
field fields
2020-06-03 15:02:15,098 INFO org.apache.flink.api.java.typeutils.TypeExtractor
[] - Class class org.apache.flink.types.Row cannot be used as a POJO
type because not all fields are valid POJO $
{code}
was (Author: zentol):
The following warning shows up if hadoop is not available. It shouldn't be a
warning, and it is not clear what is meant with "incompatible":
{code}
2020-06-03 14:44:45,520 WARN org.apache.flink.runtime.security.SecurityUtils
[] - Unable to install incompatible security context factory
org.apache.flink.runtime.security.contexts.HadoopSecurityContextFactory
{code}
Then we got some seemingly redundant messages:
{code}
2020-06-03 14:50:41,467 INFO
org.apache.flink.runtime.io.network.netty.NettyClient [] - Transport
type 'auto': using EPOLL.
2020-06-03 14:50:41,472 INFO
org.apache.flink.runtime.io.network.netty.NettyClient [] - Successful
initialization (took 127 ms).
2020-06-03 14:50:41,483 INFO
org.apache.flink.runtime.io.network.netty.NettyServer [] - Transport
type 'auto': using EPOLL.
2020-06-03 14:50:41,550 INFO
org.apache.flink.runtime.io.network.netty.NettyServer [] - Successful
initialization (took 71 ms). Listening on SocketAddress
/0:0:0:0:0:0:0:0%0:36063.
{code}
This one occurring during task executor startup is a bit unclear:
{code}
2020-06-03 14:50:39,367 INFO org.apache.flink.runtime.net.ConnectionUtils
[] - Retrieved new target address localhost/127.0.0.1:6123.
{code}
These could probably be debug messages:
{code}
2020-06-03 14:50:41,571 INFO
org.apache.flink.runtime.taskexecutor.TaskManagerConfiguration [] - Messages
have a max timeout of 10000 ms
...
2020-06-03 14:50:41,945 INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Registration
at ResourceManager attempt 1 (timeout=100ms)
2020-06-03 14:50:42,093 INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Registration
at ResourceManager attempt 2 (timeout=200ms)
{code}
When the first TaskManager registration times out you get this INFO message,
which somewhat comes out of nowhere and doesn't give much valuable information;
could probably be DEBUG instead:
{code}
020-06-03 14:50:42,109 INFO
org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] -
Ignoring outdated TaskExecutorGateway connection.
{code}
Running a simple streaming wordcount resulted in this smelly message:
{code}
2020-06-03 14:58:33,870 INFO
org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl [] -
JobManagerRunner already shutdown.
{code}
These filesystem messages are really noisy:
{code}
2020-06-03 14:58:32,994 INFO org.apache.flink.runtime.taskmanager.Task
[] - Creating FileSystem stream leak safety net for task Keyed
Aggregation -> Sink: Print to Std. Out (1/1) (7f2$
2020-06-03 14:58:33,662 INFO org.apache.flink.runtime.taskmanager.Task
[] - Ensuring all FileSystem streams are closed for task Source:
Collection Source -> Flat Map (1/1) (12bf05d6f3$
{code}
Table programs log several repeated messages:
{code}
2020-06-03 15:02:11,982 INFO org.apache.flink.table.module.ModuleManager
[] - Got FunctionDefinition 'as' from 'core' module.
2020-06-03 15:02:11,988 INFO org.apache.flink.table.module.ModuleManager
[] - Got FunctionDefinition 'sum' from 'core' module.
2020-06-03 15:02:12,139 INFO org.apache.flink.table.module.ModuleManager
[] - Got FunctionDefinition 'as' from 'core' module.
2020-06-03 15:02:12,159 INFO org.apache.flink.table.module.ModuleManager
[] - Got FunctionDefinition 'equals' from 'core' module.
2020-06-03 15:02:14,932 INFO org.apache.flink.api.java.typeutils.TypeExtractor
[] - class org.apache.flink.types.Row does not contain a getter for
field fields
2020-06-03 15:02:14,932 INFO org.apache.flink.api.java.typeutils.TypeExtractor
[] - class org.apache.flink.types.Row does not contain a setter for
field fields
2020-06-03 15:02:14,932 INFO org.apache.flink.api.java.typeutils.TypeExtractor
[] - Class class org.apache.flink.types.Row cannot be used as a POJO
type because not all fields are valid POJO $
2020-06-03 15:02:15,073 INFO org.apache.flink.api.java.typeutils.TypeExtractor
[] - class org.apache.flink.types.Row does not contain a getter for
field fields
2020-06-03 15:02:15,073 INFO org.apache.flink.api.java.typeutils.TypeExtractor
[] - class org.apache.flink.types.Row does not contain a setter for
field fields
2020-06-03 15:02:15,073 INFO org.apache.flink.api.java.typeutils.TypeExtractor
[] - Class class org.apache.flink.types.Row cannot be used as a POJO
type because not all fields are valid POJO $
2020-06-03 15:02:15,098 INFO org.apache.flink.api.java.typeutils.TypeExtractor
[] - class org.apache.flink.types.Row does not contain a getter for
field fields
2020-06-03 15:02:15,098 INFO org.apache.flink.api.java.typeutils.TypeExtractor
[] - class org.apache.flink.types.Row does not contain a setter for
field fields
2020-06-03 15:02:15,098 INFO org.apache.flink.api.java.typeutils.TypeExtractor
[] - Class class org.apache.flink.types.Row cannot be used as a POJO
type because not all fields are valid POJO $
{code}
> Check log sanity
> ----------------
>
> Key: FLINK-17977
> URL: https://issues.apache.org/jira/browse/FLINK-17977
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
> Affects Versions: 1.11.0
> Reporter: Till Rohrmann
> Priority: Critical
> Labels: release-testing
> Fix For: 1.11.0
>
>
> Run a normal Flink workload (e.g. job with fixed number of failures on
> session cluster) and check that the produced Flink logs make sense and don't
> contain confusing statements.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
