Agreed; it seems like a lot of what's needed is already in place. Would you recommend I file an issue at this point?
I completely agree that bumping the default to info is the best path forward (and yes, you're correct on the goal). I figured changing the default might be considered a "breaking" change, so a new level avoided that. Either way is perfectly fine by me! Thanks for the feedback! Isaac Whitfield On Friday, December 8, 2017 at 12:43:12 AM UTC-8, José Valim wrote: > > The minimum we can do for now is to allow compile time level purging per > application. This means the work is all done at compile time and there is > no runtime overhead. This fits well with issue #7046 > <https://github.com/elixir-lang/elixir/issues/7046> which aims to make > all logger calls purgeable without side-effects. > > In regards to adding a new level, I still don't believe it would add > anything new. If the goal is to reduce the default output, then I would > rather bump the default level to info than introduce a new level. > > > > *José Valimwww.plataformatec.com.br > <http://www.plataformatec.com.br/>Founder and Director of R&D* > > On Fri, Dec 8, 2017 at 12:28 AM, David Antaramian <[email protected] > <javascript:>> wrote: > >> I agree with Paul that this is something that should be solved at the >> Logger level and not the backend level for consistency purposes. I'm >> responsible for maintaining a backend. The backend *could* allow users >> to pass a list of applications and keep/evict matching logs based on the >> application metadata key available on each log event. But having this as >> custom logic doesn't make sense because it seems to be a more general >> concern. >> >> Approaching this from a different angle, one of the major differences >> between Elixir's Logger and logging packages in other languages is that the >> Logger is a singleton (for lack of a better term). For example, in Java you >> could instantiate a Logger instance for a specific class and then manage >> that logger and its level separate from others. >> >> To throw the idea out there, would it be more useful to be able to create >> a named Logger process separate from the main Logger process? I don't have >> a concrete idea for how that would interface with the rest of the logging >> pipeline in Elixir. My thought, though, is that only events from the main >> Logger process would be sent to backends by default; named Logger processes >> would have to be "opt-in" via explicit configuration. >> >> The additional benefit of this is that I could run the different named >> Logger processes at different levels, similar to Paul's namespace >> suggestion above. For example, I could set the named :cachex logger to >> run at :info by default. If I have an issue, I could switch it to run at >> :debug to gather diagnostic logs without the other running loggers also >> switching to :debug (and inundating me with a ton of other logs I don't >> need). >> >> - David >> >> On Thursday, December 7, 2017 at 5:43:52 PM UTC-5, Isaac Whitfield wrote: >>> >>> Hey Paul, >>> >>> Using the module is actually pretty compelling; I guess we could even >>> reproduce the application just by using the parent module of an >>> application? So we would support both with a single syntax (no need to >>> check for module vs. application). I would like that solution too; I'm just >>> not sure if we'd want to stick to the OTP style here with applications, or >>> go for the more Elixir-y route of modules? >>> >>> I completely agree that it grants a lot of control over your logging >>> without too much churn in the Logger itself (as far as I know). Like you, I >>> came from JVM languages where I'm totally used to having this readily >>> available to me; I found it quite surprising that there's nothing in the >>> Logger as it exists for it. What do you think about the extra log level >>> which exists below the default log level? >>> >>> Thank you both for input so far! >>> Isaac >>> >>> On Thursday, December 7, 2017 at 2:21:00 PM UTC-8, Paul Schoenfelder >>> wrote: >>>> >>>> I personally think that it would be nice to be able to override the log >>>> level for a specific application and/or namespace (i.e. something like a >>>> pattern, where setting the level for `Foo.Bar` will set the level for logs >>>> produced by any module starting with `Foo.Bar`). This provides control to >>>> silence noisy logging on a per-application or per-module level, for >>>> example >>>> when a dependency logs a lot of unnecessary information at too high of a >>>> log level (i.e. debug info at the info level), and you want to strip just >>>> the annoying bits; it also provides the ability to log more verbosely from >>>> a component without opening the floodgates by verbosely logging from >>>> everything in the system. >>>> >>>> While a custom logger backend could probably accomplish some or all of >>>> this, what about when you are already using multiple backends (say console >>>> and file) which both require this filtering? You either need to write a >>>> filter (and I'm not sure if that's practical here, as it seems like it >>>> might incur significant overhead, but I haven't looked into it in much >>>> detail) or reimplement all the backends you are using to do this >>>> filtering. >>>> >>>> This seems like pretty basic functionality that should be present in >>>> Logger itself, and since the pieces are already there, it's really a >>>> matter >>>> of putting them together. Having spent many years in .NET and JVM >>>> languages, the ability to specify backends, log levels, and even format on >>>> a per-appdomain/namespace/class basis was pretty much table stakes in >>>> loggers in those languages - I see no reason why we shouldn't have that >>>> capability in Elixir. >>>> >>>> Paul >>>> >>>> >>>> On Thu, Dec 7, 2017 at 2:56 PM, Isaac Whitfield <[email protected]> >>>> wrote: >>>> >>>>> Using "abusing OTP internals" makes me think that we should avoid that >>>>> bit; but I'm admittedly unfamiliar with how the backends work. When I say >>>>> "application" above, I'm mainly using it to mean modules that belong to >>>>> the >>>>> application - not processes. This would mean that GenServers started >>>>> using >>>>> application modules would also be included in the definitions. >>>>> >>>>> I think this would be more expected; what do you think? >>>>> >>>>> Isaac Whitfield >>>>> >>>>> On Thursday, December 7, 2017 at 11:18:36 AM UTC-8, Michał Muskała >>>>> wrote: >>>>>> >>>>>> There's always a question of what this setting would actually mean - >>>>>> should it affect logging from modules that belong to that application or >>>>>> should it affect logging from processes belonging to that application. >>>>>> If >>>>>> the later, it's already possible with a custom logger backend by >>>>>> leveraging >>>>>> the group leader mechanism and abusing OTP internals a little bit. >>>>>> >>>>>> Each log has the pid of the originating process - using >>>>>> :application.get_application(pid) one can obtain the application where >>>>>> this >>>>>> process runs. The backend can dispatch the log based on the >>>>>> configuration >>>>>> and only log messages from some of the applications. >>>>>> >>>>>> Michał. >>>>>> >>>>>> On 7 Dec 2017, 20:01 +0100, Isaac Whitfield <[email protected]>, wrote: >>>>>> >>>>>> This is based on previous conversation in >>>>>> https://groups.google.com/d/msg/elixir-lang-core/C5-ixZVau3U/Gkb4PyOOAQAJ, >>>>>> >>>>>> but this thread proposes an actual solution which I believe will benefit >>>>>> everyone. >>>>>> >>>>>> The Logger in Elixir is widely used due to being in the standard >>>>>> library, but misses out on a few key use cases that are familiar for >>>>>> developers coming from other languages. For one, I believe it's time to >>>>>> add >>>>>> functionality to suppress logs on a per-application basis. As the >>>>>> application is known to the logging macros it can easily filter out at >>>>>> compile time. This can also be a backwards-compatible option inside the >>>>>> Logger configuration to override the default log level on a per >>>>>> application >>>>>> basis: >>>>>> >>>>>> >>>>>> config :logger, :my_logger, >>>>>> level: :info, >>>>>> applications: [ >>>>>> library1: :debug >>>>>> ] >>>>>> >>>>>> >>>>>> This allows developers to allow logs to be noisier for specific >>>>>> applications (such as their own), without being swamped with noise from >>>>>> other applications. The :applications key will just override the default >>>>>> level provided in the :level key. If the :applications key is not ideal, >>>>>> we >>>>>> can name it something else (:apps, :override, etc). We could even just >>>>>> combine the two in :levels and use a :default for the default level. >>>>>> >>>>>> In addition to these changes, I propose that a new logging level be >>>>>> added which is lower than the default log level of :debug, with the >>>>>> intent >>>>>> that all developers log to this unless there is explicit reason to have >>>>>> their logs seen by default. The combination of this and the above >>>>>> :applications key allows developers to easily turn on logs from >>>>>> applications they're interested in at the time. It allows the >>>>>> application >>>>>> developer to decide what is important enough for another developer to >>>>>> see, >>>>>> rather than the developer having to (typically) guess. I propose that >>>>>> this >>>>>> new logging level should be named any of :trace, :verbose or :quiet >>>>>> (with >>>>>> :trace being preferred for brevity/familiarity). >>>>>> >>>>>> There are many other languages which encourage this style of logging. >>>>>> Here are a few I know of: >>>>>> >>>>>> - Node.js has the "debug" package, which is widely popular as it >>>>>> allows users to opt-in to seeing logs from specific namespaces (which >>>>>> roughly translates to OTP applications in our use case). >>>>>> - Java has SLF4J which hides debug logs by default, allowing you to >>>>>> opt in on package or class level (also hugely popular). >>>>>> - Rust's official logging framework defaults to actually hiding all >>>>>> logging, making you explicitly ask to see logs (of any kind). >>>>>> >>>>>> The interesting thing with the packages from Node.s/Java is that >>>>>> they're libraries and not in the standard library; they evolved because >>>>>> of >>>>>> the standard logging missing this stuff (although to be fair, Node.js >>>>>> only >>>>>> has console). As the Logger is so widely used in Elixir, it feels that >>>>>> we're already at the point where this should be supported. Making this >>>>>> an >>>>>> external library in Elixir would be wasted effort as we would then have >>>>>> logging from different sources with different behaviour, thus lowering >>>>>> the >>>>>> utility. I think that including this support in Logger will make logging >>>>>> feel more familiar to developers coming from the languages above, >>>>>> without >>>>>> making it unfamiliar to (e.g.) Ruby developers where the behaviour >>>>>> matches >>>>>> what Elixir currently has. >>>>>> >>>>>> I appreciate any support and/or suggestions with this. I feel >>>>>> strongly about not having a "quiet" log level, as it causes a very >>>>>> painful >>>>>> balance for a library developer when determining how verbose to be with >>>>>> logging. >>>>>> >>>>>> Thanks in advance for all replies, feel free to ask questions! >>>>>> IW >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "elixir-lang-core" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/elixir-lang-core/606029ea-9855-44c6-ba94-69113ffed45c%40googlegroups.com >>>>>> >>>>>> <https://groups.google.com/d/msgid/elixir-lang-core/606029ea-9855-44c6-ba94-69113ffed45c%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "elixir-lang-core" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/elixir-lang-core/61adb36f-e448-46c2-abf9-b1a5757a14d3%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/elixir-lang-core/61adb36f-e448-46c2-abf9-b1a5757a14d3%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >> You received this message because you are subscribed to the Google Groups >> "elixir-lang-core" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elixir-lang-core/ddf53d78-124b-4adb-8f6f-6deb5a52d0eb%40googlegroups.com >> >> <https://groups.google.com/d/msgid/elixir-lang-core/ddf53d78-124b-4adb-8f6f-6deb5a52d0eb%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "elixir-lang-core" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/bf2e48bc-6cfe-418d-b907-c1cfab0782a2%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
