[GitHub] flink pull request #5712: [FLINK-9011] YarnResourceManager spamming log file...
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/5712 ---
[GitHub] flink pull request #5712: [FLINK-9011] YarnResourceManager spamming log file...
Github user yanghua commented on a diff in the pull request: https://github.com/apache/flink/pull/5712#discussion_r176048790 --- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/Utils.java --- @@ -351,16 +351,16 @@ static ContainerLaunchContext createTaskExecutorContext( require(yarnClientUsername != null, "Environment variable %s not set", YarnConfigKeys.ENV_HADOOP_USER_NAME); final String remoteKeytabPath = env.get(YarnConfigKeys.KEYTAB_PATH); - log.info("TM:remote keytab path obtained {}", remoteKeytabPath); - final String remoteKeytabPrincipal = env.get(YarnConfigKeys.KEYTAB_PRINCIPAL); - log.info("TM:remote keytab principal obtained {}", remoteKeytabPrincipal); - final String remoteYarnConfPath = env.get(YarnConfigKeys.ENV_YARN_SITE_XML_PATH); - log.info("TM:remote yarn conf path obtained {}", remoteYarnConfPath); - final String remoteKrb5Path = env.get(YarnConfigKeys.ENV_KRB5_PATH); - log.info("TM:remote krb5 path obtained {}", remoteKrb5Path); + + if (log.isDebugEnabled()) { --- End diff -- I know as I said in the previous comment `and each of them would do the same judgement inside the debug method.`. Here I do a judgement to avoid that if the log level upper then DEBUG, it could just jump these code. ---
[GitHub] flink pull request #5712: [FLINK-9011] YarnResourceManager spamming log file...
Github user yew1eb commented on a diff in the pull request: https://github.com/apache/flink/pull/5712#discussion_r176047433 --- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/Utils.java --- @@ -351,16 +351,16 @@ static ContainerLaunchContext createTaskExecutorContext( require(yarnClientUsername != null, "Environment variable %s not set", YarnConfigKeys.ENV_HADOOP_USER_NAME); final String remoteKeytabPath = env.get(YarnConfigKeys.KEYTAB_PATH); - log.info("TM:remote keytab path obtained {}", remoteKeytabPath); - final String remoteKeytabPrincipal = env.get(YarnConfigKeys.KEYTAB_PRINCIPAL); - log.info("TM:remote keytab principal obtained {}", remoteKeytabPrincipal); - final String remoteYarnConfPath = env.get(YarnConfigKeys.ENV_YARN_SITE_XML_PATH); - log.info("TM:remote yarn conf path obtained {}", remoteYarnConfPath); - final String remoteKrb5Path = env.get(YarnConfigKeys.ENV_KRB5_PATH); - log.info("TM:remote krb5 path obtained {}", remoteKrb5Path); + + if (log.isDebugEnabled()) { --- End diff -- Sorry, I forgot to tell you that the `log.debug()` method internally calls `islogEnabled()`. You can see the `org.slf4j.Logger`'s log4j adapter class: ``` //org.slf4j.impl.Log4jLoggerAdapter.java ... public void debug(String format, Object arg) { if (logger.isDebugEnabled()) { FormattingTuple ft = MessageFormatter.format(format, arg); logger.log(FQCN, Level.DEBUG, ft.getMessage(), ft.getThrowable()); } } ... ``` ---
[GitHub] flink pull request #5712: [FLINK-9011] YarnResourceManager spamming log file...
Github user yanghua commented on a diff in the pull request: https://github.com/apache/flink/pull/5712#discussion_r175969652 --- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/Utils.java --- @@ -351,16 +351,16 @@ static ContainerLaunchContext createTaskExecutorContext( require(yarnClientUsername != null, "Environment variable %s not set", YarnConfigKeys.ENV_HADOOP_USER_NAME); final String remoteKeytabPath = env.get(YarnConfigKeys.KEYTAB_PATH); - log.info("TM:remote keytab path obtained {}", remoteKeytabPath); - final String remoteKeytabPrincipal = env.get(YarnConfigKeys.KEYTAB_PRINCIPAL); - log.info("TM:remote keytab principal obtained {}", remoteKeytabPrincipal); - final String remoteYarnConfPath = env.get(YarnConfigKeys.ENV_YARN_SITE_XML_PATH); - log.info("TM:remote yarn conf path obtained {}", remoteYarnConfPath); - final String remoteKrb5Path = env.get(YarnConfigKeys.ENV_KRB5_PATH); - log.info("TM:remote krb5 path obtained {}", remoteKrb5Path); + + if (log.isDebugEnabled()) { --- End diff -- @yew1eb , In a way, I agree with you. If there is just one `log.debug('xxx')` and no string concatination, the `if (log.isDebugEnabled())` is not necessary. But this case, there are four `log.debug()` call , and each of them would do the same judgement inside the `debug` method. So wrapping a outer judgement is performance reason. ---
[GitHub] flink pull request #5712: [FLINK-9011] YarnResourceManager spamming log file...
Github user yew1eb commented on a diff in the pull request: https://github.com/apache/flink/pull/5712#discussion_r175842877 --- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/Utils.java --- @@ -351,16 +351,16 @@ static ContainerLaunchContext createTaskExecutorContext( require(yarnClientUsername != null, "Environment variable %s not set", YarnConfigKeys.ENV_HADOOP_USER_NAME); final String remoteKeytabPath = env.get(YarnConfigKeys.KEYTAB_PATH); - log.info("TM:remote keytab path obtained {}", remoteKeytabPath); - final String remoteKeytabPrincipal = env.get(YarnConfigKeys.KEYTAB_PRINCIPAL); - log.info("TM:remote keytab principal obtained {}", remoteKeytabPrincipal); - final String remoteYarnConfPath = env.get(YarnConfigKeys.ENV_YARN_SITE_XML_PATH); - log.info("TM:remote yarn conf path obtained {}", remoteYarnConfPath); - final String remoteKrb5Path = env.get(YarnConfigKeys.ENV_KRB5_PATH); - log.info("TM:remote krb5 path obtained {}", remoteKrb5Path); + + if (log.isDebugEnabled()) { --- End diff -- the {{isDebugEnabled}} is typically used to avoid unnecessary String concatination. but checking for debugging enabled is not necessary here. ---
[GitHub] flink pull request #5712: [FLINK-9011] YarnResourceManager spamming log file...
GitHub user yanghua opened a pull request: https://github.com/apache/flink/pull/5712 [FLINK-9011] YarnResourceManager spamming log file at INFO level ## What is the purpose of the change *This pull request changed some log's level* ## Brief change log - *changed some log's level in `YarnResourceManager` and `Utils`* ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / **not documented**) You can merge this pull request into a Git repository by running: $ git pull https://github.com/yanghua/flink FLINK-9011 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/5712.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5712 commit c2c45ba436842505ac602c3f684f058af18d00c7 Author: yanghuaDate: 2018-03-17T02:54:41Z [FLINK-9011] YarnResourceManager spamming log file at INFO level ---