The minimum we can do for now is to allow compile time level purging per application. This means the work is all done at compile time and there is no runtime overhead. This fits well with issue #7046 <https://github.com/elixir-lang/elixir/issues/7046> which aims to make all logger calls purgeable without side-effects.
In regards to adding a new level, I still don't believe it would add anything new. If the goal is to reduce the default output, then I would rather bump the default level to info than introduce a new level. *José Valimwww.plataformatec.com.br <http://www.plataformatec.com.br/>Founder and Director of R&D* On Fri, Dec 8, 2017 at 12:28 AM, David Antaramian <[email protected]> wrote: > I agree with Paul that this is something that should be solved at the > Logger level and not the backend level for consistency purposes. I'm > responsible for maintaining a backend. The backend *could* allow users to > pass a list of applications and keep/evict matching logs based on the > application metadata key available on each log event. But having this as > custom logic doesn't make sense because it seems to be a more general > concern. > > Approaching this from a different angle, one of the major differences > between Elixir's Logger and logging packages in other languages is that the > Logger is a singleton (for lack of a better term). For example, in Java you > could instantiate a Logger instance for a specific class and then manage > that logger and its level separate from others. > > To throw the idea out there, would it be more useful to be able to create > a named Logger process separate from the main Logger process? I don't have > a concrete idea for how that would interface with the rest of the logging > pipeline in Elixir. My thought, though, is that only events from the main > Logger process would be sent to backends by default; named Logger processes > would have to be "opt-in" via explicit configuration. > > The additional benefit of this is that I could run the different named > Logger processes at different levels, similar to Paul's namespace > suggestion above. For example, I could set the named :cachex logger to > run at :info by default. If I have an issue, I could switch it to run at > :debug to gather diagnostic logs without the other running loggers also > switching to :debug (and inundating me with a ton of other logs I don't > need). > > - David > > On Thursday, December 7, 2017 at 5:43:52 PM UTC-5, Isaac Whitfield wrote: >> >> Hey Paul, >> >> Using the module is actually pretty compelling; I guess we could even >> reproduce the application just by using the parent module of an >> application? So we would support both with a single syntax (no need to >> check for module vs. application). I would like that solution too; I'm just >> not sure if we'd want to stick to the OTP style here with applications, or >> go for the more Elixir-y route of modules? >> >> I completely agree that it grants a lot of control over your logging >> without too much churn in the Logger itself (as far as I know). Like you, I >> came from JVM languages where I'm totally used to having this readily >> available to me; I found it quite surprising that there's nothing in the >> Logger as it exists for it. What do you think about the extra log level >> which exists below the default log level? >> >> Thank you both for input so far! >> Isaac >> >> On Thursday, December 7, 2017 at 2:21:00 PM UTC-8, Paul Schoenfelder >> wrote: >>> >>> I personally think that it would be nice to be able to override the log >>> level for a specific application and/or namespace (i.e. something like a >>> pattern, where setting the level for `Foo.Bar` will set the level for logs >>> produced by any module starting with `Foo.Bar`). This provides control to >>> silence noisy logging on a per-application or per-module level, for example >>> when a dependency logs a lot of unnecessary information at too high of a >>> log level (i.e. debug info at the info level), and you want to strip just >>> the annoying bits; it also provides the ability to log more verbosely from >>> a component without opening the floodgates by verbosely logging from >>> everything in the system. >>> >>> While a custom logger backend could probably accomplish some or all of >>> this, what about when you are already using multiple backends (say console >>> and file) which both require this filtering? You either need to write a >>> filter (and I'm not sure if that's practical here, as it seems like it >>> might incur significant overhead, but I haven't looked into it in much >>> detail) or reimplement all the backends you are using to do this filtering. >>> >>> This seems like pretty basic functionality that should be present in >>> Logger itself, and since the pieces are already there, it's really a matter >>> of putting them together. Having spent many years in .NET and JVM >>> languages, the ability to specify backends, log levels, and even format on >>> a per-appdomain/namespace/class basis was pretty much table stakes in >>> loggers in those languages - I see no reason why we shouldn't have that >>> capability in Elixir. >>> >>> Paul >>> >>> >>> On Thu, Dec 7, 2017 at 2:56 PM, Isaac Whitfield <[email protected]> wrote: >>> >>>> Using "abusing OTP internals" makes me think that we should avoid that >>>> bit; but I'm admittedly unfamiliar with how the backends work. When I say >>>> "application" above, I'm mainly using it to mean modules that belong to the >>>> application - not processes. This would mean that GenServers started using >>>> application modules would also be included in the definitions. >>>> >>>> I think this would be more expected; what do you think? >>>> >>>> Isaac Whitfield >>>> >>>> On Thursday, December 7, 2017 at 11:18:36 AM UTC-8, Michał Muskała >>>> wrote: >>>>> >>>>> There's always a question of what this setting would actually mean - >>>>> should it affect logging from modules that belong to that application or >>>>> should it affect logging from processes belonging to that application. If >>>>> the later, it's already possible with a custom logger backend by >>>>> leveraging >>>>> the group leader mechanism and abusing OTP internals a little bit. >>>>> >>>>> Each log has the pid of the originating process - using >>>>> :application.get_application(pid) one can obtain the application >>>>> where this process runs. The backend can dispatch the log based on the >>>>> configuration and only log messages from some of the applications. >>>>> >>>>> Michał. >>>>> >>>>> On 7 Dec 2017, 20:01 +0100, Isaac Whitfield <[email protected]>, wrote: >>>>> >>>>> This is based on previous conversation in >>>>> https://groups.google.com/d/msg/elixir-lang-core/C5-ixZVau3U >>>>> /Gkb4PyOOAQAJ, but this thread proposes an actual solution which I >>>>> believe will benefit everyone. >>>>> >>>>> The Logger in Elixir is widely used due to being in the standard >>>>> library, but misses out on a few key use cases that are familiar for >>>>> developers coming from other languages. For one, I believe it's time to >>>>> add >>>>> functionality to suppress logs on a per-application basis. As the >>>>> application is known to the logging macros it can easily filter out at >>>>> compile time. This can also be a backwards-compatible option inside the >>>>> Logger configuration to override the default log level on a per >>>>> application >>>>> basis: >>>>> >>>>> >>>>> config :logger, :my_logger, >>>>> level: :info, >>>>> applications: [ >>>>> library1: :debug >>>>> ] >>>>> >>>>> >>>>> This allows developers to allow logs to be noisier for specific >>>>> applications (such as their own), without being swamped with noise from >>>>> other applications. The :applications key will just override the default >>>>> level provided in the :level key. If the :applications key is not ideal, >>>>> we >>>>> can name it something else (:apps, :override, etc). We could even just >>>>> combine the two in :levels and use a :default for the default level. >>>>> >>>>> In addition to these changes, I propose that a new logging level be >>>>> added which is lower than the default log level of :debug, with the intent >>>>> that all developers log to this unless there is explicit reason to have >>>>> their logs seen by default. The combination of this and the above >>>>> :applications key allows developers to easily turn on logs from >>>>> applications they're interested in at the time. It allows the application >>>>> developer to decide what is important enough for another developer to see, >>>>> rather than the developer having to (typically) guess. I propose that this >>>>> new logging level should be named any of :trace, :verbose or :quiet (with >>>>> :trace being preferred for brevity/familiarity). >>>>> >>>>> There are many other languages which encourage this style of logging. >>>>> Here are a few I know of: >>>>> >>>>> - Node.js has the "debug" package, which is widely popular as it >>>>> allows users to opt-in to seeing logs from specific namespaces (which >>>>> roughly translates to OTP applications in our use case). >>>>> - Java has SLF4J which hides debug logs by default, allowing you to >>>>> opt in on package or class level (also hugely popular). >>>>> - Rust's official logging framework defaults to actually hiding all >>>>> logging, making you explicitly ask to see logs (of any kind). >>>>> >>>>> The interesting thing with the packages from Node.s/Java is that >>>>> they're libraries and not in the standard library; they evolved because of >>>>> the standard logging missing this stuff (although to be fair, Node.js only >>>>> has console). As the Logger is so widely used in Elixir, it feels that >>>>> we're already at the point where this should be supported. Making this an >>>>> external library in Elixir would be wasted effort as we would then have >>>>> logging from different sources with different behaviour, thus lowering the >>>>> utility. I think that including this support in Logger will make logging >>>>> feel more familiar to developers coming from the languages above, without >>>>> making it unfamiliar to (e.g.) Ruby developers where the behaviour matches >>>>> what Elixir currently has. >>>>> >>>>> I appreciate any support and/or suggestions with this. I feel strongly >>>>> about not having a "quiet" log level, as it causes a very painful balance >>>>> for a library developer when determining how verbose to be with logging. >>>>> >>>>> Thanks in advance for all replies, feel free to ask questions! >>>>> IW >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "elixir-lang-core" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/elixir-lang-core/606029ea- >>>>> 9855-44c6-ba94-69113ffed45c%40googlegroups.com >>>>> <https://groups.google.com/d/msgid/elixir-lang-core/606029ea-9855-44c6-ba94-69113ffed45c%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "elixir-lang-core" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit https://groups.google.com/d/ms >>>> gid/elixir-lang-core/61adb36f-e448-46c2-abf9-b1a5757a14d3%40 >>>> googlegroups.com >>>> <https://groups.google.com/d/msgid/elixir-lang-core/61adb36f-e448-46c2-abf9-b1a5757a14d3%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- > You received this message because you are subscribed to the Google Groups > "elixir-lang-core" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit https://groups.google.com/d/ms > gid/elixir-lang-core/ddf53d78-124b-4adb-8f6f-6deb5a52d0eb% > 40googlegroups.com > <https://groups.google.com/d/msgid/elixir-lang-core/ddf53d78-124b-4adb-8f6f-6deb5a52d0eb%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elixir-lang-core" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4%2BGvCEV1qCQNL9NLJjfxbzVSknqyZdij9j%2By%3DXLkbnFhw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
