It seems that we have reached a consensus that the proposal will not be implemented in Flink. I will mark the FLIP as discarded if there are no objections.
Thanks, everyone, for joining the discussion again! Wenhao On Tue, Jan 11, 2022 at 11:12 PM Wenhao Ji <predator....@gmail.com> wrote: > > Hi all, > > Yes, indeed. > After I did some investigation on similar features provided by the > Cloud platforms, I actually found several popular Clouds have already > offered this. > > - AWS Kinesis: Setting the Application Logging Level [1], which is > implemented by UpdateApplication API [2]. > - Ververica: Logging & Metrics[3], by changing the template. > - Alicloud: Configure job logs [4], which is quite similar to > Ververica also by changing the template > - Cloudera: Enabling Flink DEBUG logging[5], by changing the > Configuration and triggering a restart > > It looks like this feature is not necessary. It has been developed in > one way or another by so many platforms in the ecosystem. > > [1]: > https://docs.aws.amazon.com/kinesisanalytics/latest/java/cloudwatch-logs.html#cloudwatch-level > [2]: > https://docs.aws.amazon.com/kinesisanalytics/latest/apiv2/API_UpdateApplication.html > [3]: https://docs.ververica.com/platform_operations/logging_metrics.html > [4]: https://www.alibabacloud.com/help/doc-detail/173646.htm#title-1ay-hju-pka > [5]: > https://docs.cloudera.com/csa/1.6.0/monitoring/topics/csa-ssb-enabling-debug-logging.html > > Thanks, > Wenhao > > On Tue, Jan 11, 2022 at 8:24 PM Martijn Visser <mart...@ververica.com> wrote: > > > > Hi all, > > > > I agree with Konstantin, this feels like a problem that shouldn't be solved > > via Apache Flink but via the logging ecosystem itself. > > > > Best regards, > > > > Martijn > > > > On Tue, 11 Jan 2022 at 13:11, Konstantin Knauf <kna...@apache.org> wrote: > > > > > I've now read over the discussion on the ticket, and I am personally not > > > in > > > favor of adding this functionality to Flink via the REST API or Web UI. I > > > believe that changing the logging configuration via the existing > > > configuration files (log4j or logback) is good enough, to justify not > > > increasing the scope of Flink in that direction. As you specifically > > > mention YARN: doesn't Cloudera's Hadoop platform, for example, offer means > > > to manage the configuration files for all worker nodes from a central > > > configuration management system? It overall feels like we are trying to > > > solve a problem in Apache Flink that is already solved in its ecosystem > > > and > > > increases the scope of the project without adding core value. I also > > > expect > > > that over time the exposed logging configuration options would become more > > > and more complex. > > > > > > I am curious to hear what others think. > > > > > > On Tue, Jan 11, 2022 at 10:34 AM Chesnay Schepler <ches...@apache.org> > > > wrote: > > > > > > > Reloading the config from the filesystem is already enabled by default; > > > > that was one of the things that made us switch to Log4j 2. > > > > > > > > The core point of contention w.r.t. this topic is whether having the > > > > admin ssh into the machine is too inconvenient. > > > > > > > > Personally I still think that the the current capabilities are > > > > sufficient, and I do not want us to rely on internals of the logging > > > > backends in production code. > > > > > > > > On 10/01/2022 17:26, Konstantin Knauf wrote: > > > > > Thank you for starting the discussion. Being able to change the > > > > > logging > > > > > level at runtime is very valuable in my experience. > > > > > > > > > > Instead of introducing our own API (and eventually even persistence), > > > > could > > > > > we just periodically reload the log4j or logback configuration from > > > > > the > > > > > environment/filesystem? I only quickly googled the topic and [1,2] > > > > suggest > > > > > that this might be possible? > > > > > > > > > > [1] https://stackoverflow.com/a/16216956/6422562? > > > > > [2] https://logback.qos.ch/manual/configuration.html#autoScan > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Jan 10, 2022 at 5:10 PM Wenhao Ji <predator....@gmail.com> > > > > wrote: > > > > > > > > > >> Hi everyone, > > > > >> > > > > >> Hope you enjoyed the Holiday Season. > > > > >> > > > > >> I would like to start the discussion on the improvement purpose > > > > >> FLIP-210 [1] which aims to provide a way to change log levels at > > > > >> runtime to simplify issues and bugs detection as reported in the > > > > >> ticket FLINK-16478 [2]. > > > > >> Firstly, thanks Xingxing Di and xiaodao for their previous effort. > > > > >> The > > > > >> FLIP I drafted is largely influenced by their previous designs > > > > >> [3][4]. > > > > >> Although we have reached some agreements under the jira comments > > > > >> about > > > > >> the scope of this feature, we still have the following questions > > > > >> listed below ready to be discussed in this thread. > > > > >> > > > > >> ## Question 1 > > > > >> > > > > >>> Creating as custom DSL and implementing it for several logging > > > backend > > > > >> sounds like quite a maintenance burden. Extensions to the DSL, and > > > > >> supported backends, could become quite an effort. (by Chesnay > > > Schepler) > > > > >> > > > > >> I tried to design the API of the logging backend to stay away from > > > > >> the > > > > >> details of implementations but I did not find any slf4j-specific API > > > > >> that is available to change the log level of a logger. So what I did > > > > >> is to introduce another kind of abstraction on top of the slf4j / > > > > >> log4j / logback so that we will not depend on the logging provider's > > > > >> api directly. It will be convenient for us to adopt any other logging > > > > >> providers. Please see the "Logging Abstraction" section. > > > > >> > > > > >> ## Question 2 > > > > >> > > > > >>> Do we know whether other systems support this kind of feature? If > > > yes, > > > > >> how do they solve it for different logging backends? (by Till > > > Rohrmann) > > > > >> > > > > >> I investigated several Java frameworks including Spark, Storm, and > > > > >> Spring Boot. Here is what I found. > > > > >> Spark & Storm directly depend on the log4j implementations, which > > > > >> means they do not support any other slf4j implementation at all. They > > > > >> simply call the log4j api directly. (see SparkContext.scala#L381 [5], > > > > >> Utils.scala#L2443 [6] in Spark, and LogConfigManager.java#L144 [7] in > > > > >> Storm). They are pretty different from what Flink provides. > > > > >> However, I found Spring Boot has implemented what we are interested > > > > >> in. Just as Flink, Spring boot also supports many slf4j > > > > >> implementations. Users are not limited to log4j. They have the > > > > >> ability > > > > >> to declare different logging frameworks by importing certain > > > > >> dependencies. After that spring will decide the activated one by > > > > >> scanning its classpath and context. (see LoggingSystem.java#L164 [8] > > > > >> and LoggersEndpoint.java#L99 [9]) > > > > >> > > > > >> ## Question 3 > > > > >> > > > > >> Besides the questions raised in the jira comments, I also find > > > > >> another > > > > >> thing that has not been discussed. Considering this feature as an > > > > >> MVP, > > > > >> do we need to introduce a HighAvailabilityService to store the log > > > > >> settings so that they can be synced to newly-joined task managers and > > > > >> also job manager followers for consistency? This issue is included in > > > > >> the "Limitations" section in the flip. > > > > >> > > > > >> Finally, thanks for your time for joining this discussion and > > > > >> reviewing this FLIP. I would appreciate it if you could have any > > > > >> comments or suggestions on this. > > > > >> > > > > >> > > > > >> [1]: > > > > >> > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-210%3A+Change+logging+level+dynamically+at+runtime > > > > >> [2]: https://issues.apache.org/jira/browse/FLINK-16478 > > > > >> [3]: > > > > >> > > > > > > > https://docs.google.com/document/d/1Q02VSSBzlZaZzvxuChIo1uinw8KDQsyTZUut6_IDErY > > > > >> [4]: > > > > >> > > > > > > > https://docs.google.com/document/d/19AyuTHeERP6JKmtHYnCdBw29LnZpRkbTS7K12q4OfbA > > > > >> [5]: > > > > >> > > > > > > > https://github.com/apache/spark/blob/11596b3b17b5e0f54e104cd49b1397c33c34719d/core/src/main/scala/org/apache/spark/SparkContext.scala#L381 > > > > >> [6]: > > > > >> > > > > > > > https://github.com/apache/spark/blob/11596b3b17b5e0f54e104cd49b1397c33c34719d/core/src/main/scala/org/apache/spark/util/Utils.scala#L2433 > > > > >> [7]: > > > > >> > > > > > > > https://github.com/apache/storm/blob/3f96c249cbc17ce062491bfbb39d484e241ab168/storm-client/src/jvm/org/apache/storm/daemon/worker/LogConfigManager.java#L144 > > > > >> [8]: > > > > >> > > > > > > > https://github.com/spring-projects/spring-boot/blob/main/spring-boot-project/spring-boot/src/main/java/org/springframework/boot/logging/LoggingSystem.java#L164 > > > > >> [9]: > > > > >> > > > > > > > https://github.com/spring-projects/spring-boot/blob/main/spring-boot-project/spring-boot-actuator/src/main/java/org/springframework/boot/actuate/logging/LoggersEndpoint.java#L99 > > > > >> > > > > >> Thanks, > > > > >> Wenhao > > > > >> > > > > > > > > > > > > > > > > > > > -- > > > > > > Konstantin Knauf > > > > > > https://twitter.com/snntrable > > > > > > https://github.com/knaufk > > >
