On Wed, Dec 7, 2011 at 2:51 AM, Jason Smith <j...@iriscouch.com> wrote: > Hi, all. Iris Couch urgently needs improved logging facilities in > CouchDB. My goal is to make something we all love and get it accepted > upstream, God willing. Or committers willing. But I repeat myself! > > This is the brainstorming and requirements gathering phase. In the > CouchDB of your dreams, logging system fits you like an old pair of > sneakers. It's perfect. Now, what characteristics does that system > exhibit? I will compile feedback into a spec on the wiki. > > I hope to avoid bikeshedding. Seriously, please don't even mention a > product or project by name. At this phase I hope to stick to > descriptions of functionality, goals, and non-goals. I want to > evaluate tools later. > > To start the discussion: logging is viewed differently based on your > relationship with CouchDB: > > 1. Developers of CouchDB > 2. Administrators of a couch > 3. Application developers > > My roles are administration, and a little bit of development. > Requirements, in no order. > > * Idiomatic Erlang > > * Is a compelling place for new people to contribute. Miguel de Icaza > talks about this. It's not enough that the source code is public. You > have to provide a smooth on-ramp, where people people get great bang > for their buck. They write a modest feature, and are rewarded by > seeing its effects immediately. In other words: plugins. Or maybe a > behaviour. Or some way to swap in formatters and data sinks. I don't > want to write a Loggly target (http://loggly.com). Loggly should be > begging me to merge their module. > > * 1st cut, no change to the default behavior. You still get the that > peculiar log file you know and love. People are parsing their log > files, and might expect version 1.x not to change. > > * Existing code still works. No sweeping changes hitting every > ?LOG_INFO and ?LOG_DEBUG. > > (Filipe, would you please share your thoughts on these? I think you > struggled with the conflict between them recently.) > * No performance impact (non-blocking)... > * ... but also, impossible or difficult to overwhelm or crash or lose logs. > > (The next few points sort of fit together) > > * Logs are not strings, but data structures, with data (the log > message) and metadata (severity, line number, maybe the call stack, > etc.) > > * More log levels. Roughly: trace, debug, info, warn, error, fatal > > * Maybe automatic trace logs upon function entry/exit with return > values. Not sure if this is doable. Maybe a compile option, for `make > dev` > > * When you log something, your module, line number, and maybe PID are known > > * "Components" or categories, or tags, where multiple .erl files or > individual log events can share a common property ("http", "views", > > * A policy decides what to do with logs based on level, module, or > component. You can set the policy either via configuration or > programatically. > > * There is a formatter API to serialize log data. Built-in formatters > include the legacy format, and Jan's Apache httpd combined format. > > * There is a transport API to receive logs and DTRT. > > * I know this is insane, but kill -HUP <pid> should make couch reopen > its log files. Okay, I'll settle down now. > > = Non Goals = > > * Log rotation. I have never seen a rotation feature in an application > which was better than the OS tools. And you can get problem where both > the server and the OS are rotating the same logs. I have seen that > happen, twice. Or was it once? Of course, people could write a > rotating file transport. > > -- > Iris Couch
I fully agree, we need better logging in couch. I don't really like the idea of saving any data structures, but why not. Imo such things could be saved as binaries strings and then parse but ... Technically all of these things can be covered wby lagger I think :h https://github.com/basho/lager It handle different backend, attribute supports, logs rotation, tracing, loggers integation .. More on their page. We expect to use it in refuge, and I think I can provide a patch for couch soon as well. - benoît
