Doh, missed that you already filed one! Here's it is for everyone's reference: https://github.com/elixir-lang/elixir/issues/7095
On Friday, December 8, 2017 at 8:03:42 AM UTC-8, Isaac Whitfield wrote: > > Agreed; it seems like a lot of what's needed is already in place. Would > you recommend I file an issue at this point? > > I completely agree that bumping the default to info is the best path > forward (and yes, you're correct on the goal). I figured changing the > default might be considered a "breaking" change, so a new level avoided > that. Either way is perfectly fine by me! > > Thanks for the feedback! > Isaac Whitfield > > On Friday, December 8, 2017 at 12:43:12 AM UTC-8, José Valim wrote: >> >> The minimum we can do for now is to allow compile time level purging per >> application. This means the work is all done at compile time and there is >> no runtime overhead. This fits well with issue #7046 >> <https://github.com/elixir-lang/elixir/issues/7046> which aims to make >> all logger calls purgeable without side-effects. >> >> In regards to adding a new level, I still don't believe it would add >> anything new. If the goal is to reduce the default output, then I would >> rather bump the default level to info than introduce a new level. >> >> >> >> *José Valimwww.plataformatec.com.br >> <http://www.plataformatec.com.br/>Founder and Director of R&D* >> >> On Fri, Dec 8, 2017 at 12:28 AM, David Antaramian <[email protected]> >> wrote: >> >>> I agree with Paul that this is something that should be solved at the >>> Logger level and not the backend level for consistency purposes. I'm >>> responsible for maintaining a backend. The backend *could* allow users >>> to pass a list of applications and keep/evict matching logs based on the >>> application metadata key available on each log event. But having this >>> as custom logic doesn't make sense because it seems to be a more general >>> concern. >>> >>> Approaching this from a different angle, one of the major differences >>> between Elixir's Logger and logging packages in other languages is that the >>> Logger is a singleton (for lack of a better term). For example, in Java you >>> could instantiate a Logger instance for a specific class and then manage >>> that logger and its level separate from others. >>> >>> To throw the idea out there, would it be more useful to be able to >>> create a named Logger process separate from the main Logger process? I >>> don't have a concrete idea for how that would interface with the rest of >>> the logging pipeline in Elixir. My thought, though, is that only events >>> from the main Logger process would be sent to backends by default; named >>> Logger processes would have to be "opt-in" via explicit configuration. >>> >>> The additional benefit of this is that I could run the different named >>> Logger processes at different levels, similar to Paul's namespace >>> suggestion above. For example, I could set the named :cachex logger to >>> run at :info by default. If I have an issue, I could switch it to run >>> at :debug to gather diagnostic logs without the other running loggers >>> also switching to :debug (and inundating me with a ton of other logs I >>> don't need). >>> >>> - David >>> >>> On Thursday, December 7, 2017 at 5:43:52 PM UTC-5, Isaac Whitfield wrote: >>>> >>>> Hey Paul, >>>> >>>> Using the module is actually pretty compelling; I guess we could even >>>> reproduce the application just by using the parent module of an >>>> application? So we would support both with a single syntax (no need to >>>> check for module vs. application). I would like that solution too; I'm >>>> just >>>> not sure if we'd want to stick to the OTP style here with applications, or >>>> go for the more Elixir-y route of modules? >>>> >>>> I completely agree that it grants a lot of control over your logging >>>> without too much churn in the Logger itself (as far as I know). Like you, >>>> I >>>> came from JVM languages where I'm totally used to having this readily >>>> available to me; I found it quite surprising that there's nothing in the >>>> Logger as it exists for it. What do you think about the extra log level >>>> which exists below the default log level? >>>> >>>> Thank you both for input so far! >>>> Isaac >>>> >>>> On Thursday, December 7, 2017 at 2:21:00 PM UTC-8, Paul Schoenfelder >>>> wrote: >>>>> >>>>> I personally think that it would be nice to be able to override the >>>>> log level for a specific application and/or namespace (i.e. something >>>>> like >>>>> a pattern, where setting the level for `Foo.Bar` will set the level for >>>>> logs produced by any module starting with `Foo.Bar`). This provides >>>>> control >>>>> to silence noisy logging on a per-application or per-module level, for >>>>> example when a dependency logs a lot of unnecessary information at too >>>>> high >>>>> of a log level (i.e. debug info at the info level), and you want to strip >>>>> just the annoying bits; it also provides the ability to log more >>>>> verbosely >>>>> from a component without opening the floodgates by verbosely logging from >>>>> everything in the system. >>>>> >>>>> While a custom logger backend could probably accomplish some or all of >>>>> this, what about when you are already using multiple backends (say >>>>> console >>>>> and file) which both require this filtering? You either need to write a >>>>> filter (and I'm not sure if that's practical here, as it seems like it >>>>> might incur significant overhead, but I haven't looked into it in much >>>>> detail) or reimplement all the backends you are using to do this >>>>> filtering. >>>>> >>>>> This seems like pretty basic functionality that should be present in >>>>> Logger itself, and since the pieces are already there, it's really a >>>>> matter >>>>> of putting them together. Having spent many years in .NET and JVM >>>>> languages, the ability to specify backends, log levels, and even format >>>>> on >>>>> a per-appdomain/namespace/class basis was pretty much table stakes in >>>>> loggers in those languages - I see no reason why we shouldn't have that >>>>> capability in Elixir. >>>>> >>>>> Paul >>>>> >>>>> >>>>> On Thu, Dec 7, 2017 at 2:56 PM, Isaac Whitfield <[email protected]> >>>>> wrote: >>>>> >>>>>> Using "abusing OTP internals" makes me think that we should avoid >>>>>> that bit; but I'm admittedly unfamiliar with how the backends work. When >>>>>> I >>>>>> say "application" above, I'm mainly using it to mean modules that belong >>>>>> to >>>>>> the application - not processes. This would mean that GenServers started >>>>>> using application modules would also be included in the definitions. >>>>>> >>>>>> I think this would be more expected; what do you think? >>>>>> >>>>>> Isaac Whitfield >>>>>> >>>>>> On Thursday, December 7, 2017 at 11:18:36 AM UTC-8, Michał Muskała >>>>>> wrote: >>>>>>> >>>>>>> There's always a question of what this setting would actually mean - >>>>>>> should it affect logging from modules that belong to that application >>>>>>> or >>>>>>> should it affect logging from processes belonging to that application. >>>>>>> If >>>>>>> the later, it's already possible with a custom logger backend by >>>>>>> leveraging >>>>>>> the group leader mechanism and abusing OTP internals a little bit. >>>>>>> >>>>>>> Each log has the pid of the originating process - using >>>>>>> :application.get_application(pid) one can obtain the application where >>>>>>> this >>>>>>> process runs. The backend can dispatch the log based on the >>>>>>> configuration >>>>>>> and only log messages from some of the applications. >>>>>>> >>>>>>> Michał. >>>>>>> >>>>>>> On 7 Dec 2017, 20:01 +0100, Isaac Whitfield <[email protected]>, >>>>>>> wrote: >>>>>>> >>>>>>> This is based on previous conversation in >>>>>>> https://groups.google.com/d/msg/elixir-lang-core/C5-ixZVau3U/Gkb4PyOOAQAJ, >>>>>>> >>>>>>> but this thread proposes an actual solution which I believe will >>>>>>> benefit >>>>>>> everyone. >>>>>>> >>>>>>> The Logger in Elixir is widely used due to being in the standard >>>>>>> library, but misses out on a few key use cases that are familiar for >>>>>>> developers coming from other languages. For one, I believe it's time to >>>>>>> add >>>>>>> functionality to suppress logs on a per-application basis. As the >>>>>>> application is known to the logging macros it can easily filter out at >>>>>>> compile time. This can also be a backwards-compatible option inside the >>>>>>> Logger configuration to override the default log level on a per >>>>>>> application >>>>>>> basis: >>>>>>> >>>>>>> >>>>>>> config :logger, :my_logger, >>>>>>> level: :info, >>>>>>> applications: [ >>>>>>> library1: :debug >>>>>>> ] >>>>>>> >>>>>>> >>>>>>> This allows developers to allow logs to be noisier for specific >>>>>>> applications (such as their own), without being swamped with noise from >>>>>>> other applications. The :applications key will just override the >>>>>>> default >>>>>>> level provided in the :level key. If the :applications key is not >>>>>>> ideal, we >>>>>>> can name it something else (:apps, :override, etc). We could even just >>>>>>> combine the two in :levels and use a :default for the default level. >>>>>>> >>>>>>> In addition to these changes, I propose that a new logging level be >>>>>>> added which is lower than the default log level of :debug, with the >>>>>>> intent >>>>>>> that all developers log to this unless there is explicit reason to have >>>>>>> their logs seen by default. The combination of this and the above >>>>>>> :applications key allows developers to easily turn on logs from >>>>>>> applications they're interested in at the time. It allows the >>>>>>> application >>>>>>> developer to decide what is important enough for another developer to >>>>>>> see, >>>>>>> rather than the developer having to (typically) guess. I propose that >>>>>>> this >>>>>>> new logging level should be named any of :trace, :verbose or :quiet >>>>>>> (with >>>>>>> :trace being preferred for brevity/familiarity). >>>>>>> >>>>>>> There are many other languages which encourage this style of >>>>>>> logging. Here are a few I know of: >>>>>>> >>>>>>> - Node.js has the "debug" package, which is widely popular as it >>>>>>> allows users to opt-in to seeing logs from specific namespaces (which >>>>>>> roughly translates to OTP applications in our use case). >>>>>>> - Java has SLF4J which hides debug logs by default, allowing you to >>>>>>> opt in on package or class level (also hugely popular). >>>>>>> - Rust's official logging framework defaults to actually hiding all >>>>>>> logging, making you explicitly ask to see logs (of any kind). >>>>>>> >>>>>>> The interesting thing with the packages from Node.s/Java is that >>>>>>> they're libraries and not in the standard library; they evolved because >>>>>>> of >>>>>>> the standard logging missing this stuff (although to be fair, Node.js >>>>>>> only >>>>>>> has console). As the Logger is so widely used in Elixir, it feels that >>>>>>> we're already at the point where this should be supported. Making this >>>>>>> an >>>>>>> external library in Elixir would be wasted effort as we would then have >>>>>>> logging from different sources with different behaviour, thus lowering >>>>>>> the >>>>>>> utility. I think that including this support in Logger will make >>>>>>> logging >>>>>>> feel more familiar to developers coming from the languages above, >>>>>>> without >>>>>>> making it unfamiliar to (e.g.) Ruby developers where the behaviour >>>>>>> matches >>>>>>> what Elixir currently has. >>>>>>> >>>>>>> I appreciate any support and/or suggestions with this. I feel >>>>>>> strongly about not having a "quiet" log level, as it causes a very >>>>>>> painful >>>>>>> balance for a library developer when determining how verbose to be with >>>>>>> logging. >>>>>>> >>>>>>> Thanks in advance for all replies, feel free to ask questions! >>>>>>> IW >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "elixir-lang-core" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/elixir-lang-core/606029ea-9855-44c6-ba94-69113ffed45c%40googlegroups.com >>>>>>> >>>>>>> <https://groups.google.com/d/msgid/elixir-lang-core/606029ea-9855-44c6-ba94-69113ffed45c%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "elixir-lang-core" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/elixir-lang-core/61adb36f-e448-46c2-abf9-b1a5757a14d3%40googlegroups.com >>>>>> >>>>>> <https://groups.google.com/d/msgid/elixir-lang-core/61adb36f-e448-46c2-abf9-b1a5757a14d3%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elixir-lang-core" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elixir-lang-core/ddf53d78-124b-4adb-8f6f-6deb5a52d0eb%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/elixir-lang-core/ddf53d78-124b-4adb-8f6f-6deb5a52d0eb%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- You received this message because you are subscribed to the Google Groups "elixir-lang-core" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/8ca8d1ce-8996-4b55-923b-f8e0b99bf054%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
