[
https://issues.apache.org/jira/browse/STORM-412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14935179#comment-14935179
]
ASF GitHub Bot commented on STORM-412:
--------------------------------------
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/766#discussion_r40671792
--- Diff: storm-core/src/clj/backtype/storm/command/set_log_level.clj ---
@@ -0,0 +1,75 @@
+;; Licensed to the Apache Software Foundation (ASF) under one
+;; or more contributor license agreements. See the NOTICE file
+;; distributed with this work for additional information
+;; regarding copyright ownership. The ASF licenses this file
+;; to you under the Apache License, Version 2.0 (the
+;; "License"); you may not use this file except in compliance
+;; with the License. You may obtain a copy of the License at
+;;
+;; http://www.apache.org/licenses/LICENSE-2.0
+;;
+;; Unless required by applicable law or agreed to in writing, software
+;; distributed under the License is distributed on an "AS IS" BASIS,
+;; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+;; See the License for the specific language governing permissions and
+;; limitations under the License.
+(ns backtype.storm.command.set-log-level
+ (:use [clojure.tools.cli :only [cli]])
+ (:use [backtype.storm thrift log])
+ (:import [org.apache.logging.log4j Level])
+ (:import [backtype.storm.generated LogConfig LogLevel LogLevelAction])
+ (:gen-class))
+
+(defn- get-storm-id
+ "Get topology id for a running topology from the topology name."
+ [nimbus name]
+ (let [info (.getClusterInfo nimbus)
+ topologies (.get_topologies info)
+ topology (first (filter (fn [topo] (= name (.get_name topo)))
topologies))]
+ (if topology
+ (.get_id topology)
+ (throw (.IllegalArgumentException (str name " is not a running
topology"))))))
+
+(defn- parse-named-log-levels [action]
+ "Parses [logger name]=[level string]:[optional timeout],[logger name2]...
+
+ e.g. ROOT=DEBUG:30
+ root logger, debug for 30 seconds
+
+ org.apache.foo=WARN
+ org.apache.foo set to WARN indefinitely"
+ (fn [^String s]
+ (let [log-args (re-find #"(.*)=([A-Z]+):?(\d*)" s)
+ name (if (= action LogLevelAction/REMOVE) s (nth log-args 1))
+ level (Level/toLevel (nth log-args 2))
+ timeout-str (nth log-args 3)
+ log-level (LogLevel.)]
+ (if (= action LogLevelAction/REMOVE)
+ (.set_action log-level action)
+ (do
+ (.set_action log-level action)
+ (.set_target_log_level log-level (.toString level))
+ (.set_reset_log_level_timeout_secs log-level
+ (Integer. (if (= timeout-str "") "0" timeout-str)))))
+ {name log-level})))
+
+(defn- merge-together [previous key val]
+ (assoc previous key
+ (if-let [oldval (get previous key)]
+ (merge oldval val)
+ val)))
+
+(defn -main [& args]
+ (let [[{log-setting :log-setting remove-log-setting :remove-log-setting}
[name] _]
+ (cli args ["-l" "--log-setting"
+ :parse-fn (parse-named-log-levels
LogLevelAction/UPDATE)
+ :assoc-fn merge-together]
+ ["-r" "--remove-log-setting"
+ :parse-fn (parse-named-log-levels
LogLevelAction/REMOVE)
+ :assoc-fn merge-together])]
+ (with-configured-nimbus-connection nimbus
--- End diff --
It might be nice to move this down to just around line 75. That way we
don't establish a connection to nimbus even when there are errors, but honestly
this is very very minor, and I am OK without it.
> Allow users to modify logging levels of running topologies.
> -----------------------------------------------------------
>
> Key: STORM-412
> URL: https://issues.apache.org/jira/browse/STORM-412
> Project: Apache Storm
> Issue Type: Improvement
> Reporter: Robert Joseph Evans
>
> It really would be great to be able to turn on debug logging for different
> parts of an already running topology, and then turn them off again later. Or
> even better if they could turn it on for just a few workers, so we don't
> flood the logs for a very large topology.
> logback already has the ability to refresh its config periodically so really
> what we needs is some code to generate logback configs on the fly based off
> of the currently desired user settings, and API to modify those dynamically,
> and a way to distribute those configs/settings to the supervisors. I am not
> too concerned about persisting these settings long term. If nimbus goes down
> and they reset back to default, I think that would be OK.
> For the distribution of the configs I think it would be best to setup a
> RESTful web service that is a part of nimbus. logback already supports using
> http to download configs. The service could use the cached logging settings
> for a given topology, or individual worker and the URL of the request to
> generate a logging config on the fly for a specific worker. As for the APIs
> I think a few new thrift calls to nimbus would be good, a command line tool,
> and possibly something on the UI.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
