Re: getting rid of all-machines.log

2014-08-27 Thread Gabriel Samfira
Hi David, (some comments in-line) As a user that wants to deploy a charm on a Windows machine, I want to be able to have a local log file on that machine for the machine agent and for the units deployed to it. I also want to be able to aggregate all those logs the same way Ubuntu workloads do

Re: getting rid of all-machines.log

2014-08-27 Thread Gabriel Samfira
On 27.08.2014 08:12, John Meinel wrote: ... I may be misremembering, but at the time that was the preferred approach. But then someone said Go's inbuilt syslog APIs were broke, so the compromise was to use rsyslog forwarding. Does anyone else recall why it may have been said that Go's

Re: getting rid of all-machines.log

2014-08-27 Thread Gabriel Samfira
That one is an easy fix in any case. We are using a forked version of the syslog package. Removing the pid from the writeString() method should be trivial. Gabriel On 27.08.2014 13:45, John Meinel wrote: So at the very least our default logging *doesn't* include the PID, though the rest

Re: getting rid of all-machines.log

2014-08-26 Thread Horacio Duran
Hey, In an effort to move forward with juju's windows integration I have summarized what seems to be the core points of this discussion to the best of my ability (please excuse me if I missed or misunderstood something). The two core points of discussion on this thread are: * should we remove

Re: getting rid of all-machines.log

2014-08-26 Thread David Cheney
Hi Horatio, I don't see a way to resolve the very disparate set of opinions you've highlighted below. It's also not clear from your email who is responsible for making a decision. I suggest reframing the discussion as user stories. ie * As a Juju user with a Windows workstation I want to use

Re: getting rid of all-machines.log

2014-08-26 Thread Mark Ramm-Christensen (Canonical.com)
you'll have to be more specific, there's been a shotgun of statements in this thread, touching on logstash, aggregation removal, rsyslog removal, log rotation, deferring to stderr/stdout, 12factor apps, working with ha state servers, etc. I was referring to Nate's lumberjack package (PR

Re: getting rid of all-machines.log

2014-08-26 Thread John Meinel
My personal vote would be: a) Use something that can write directly to multiple syslog receivers over a TLS encrypted connection from inside the jujud binary (e.g. don't use rsyslog to read the log files and forward them on to the state servers, but just write directly) I'd actually like to

Re: getting rid of all-machines.log

2014-08-26 Thread Ian Booth
On 27/08/14 14:56, John Meinel wrote: My personal vote would be: a) Use something that can write directly to multiple syslog receivers over a TLS encrypted connection from inside the jujud binary (e.g. don't use rsyslog to read the log files and forward them on to the state servers, but

Re: getting rid of all-machines.log

2014-08-26 Thread David Cheney
On Wed, Aug 27, 2014 at 3:04 PM, Ian Booth ian.bo...@canonical.com wrote: On 27/08/14 14:56, John Meinel wrote: My personal vote would be: a) Use something that can write directly to multiple syslog receivers over a TLS encrypted connection from inside the jujud binary (e.g. don't use

Re: getting rid of all-machines.log

2014-08-15 Thread Kapil Thangavelu
On Fri, Aug 15, 2014 at 7:36 AM, Gabriel Samfira gsamf...@cloudbasesolutions.com wrote: I think this thread has become a bit lengthy, and we have started to loose perspective on what we are actually trying to accomplish. agreed. afaics that's how do we support logging on windows.

Re: getting rid of all-machines.log

2014-08-15 Thread Gabriel Samfira
On 15.08.2014 16:35, Kapil Thangavelu wrote: On Fri, Aug 15, 2014 at 7:36 AM, Gabriel Samfira gsamf...@cloudbasesolutions.commailto:gsamf...@cloudbasesolutions.com wrote: I think this thread has become a bit lengthy, and we have started to loose perspective on what we are actually trying to

Re: getting rid of all-machines.log

2014-08-14 Thread Gabriel Samfira
Please forgive the lengthy response :). The following are just a few thoughts regarding this subject. On 14.08.2014 06:13, David Cheney wrote: Ian asked me to post my thoughts here. I am not in favour of applications rolling their own logs, I believe that applications should not know

Re: getting rid of all-machines.log

2014-08-14 Thread Matt Rae
Many operations teams already have a standard log collecting systems. I think it would be best to be flexible enough to work in environments with existing systems. Standard ways are logging to syslog so any syslog implementation can be used, or logging to stdout so a supervisor like djb

Re: getting rid of all-machines.log

2014-08-14 Thread Gustavo Niemeyer
On Thu, Aug 14, 2014 at 1:35 PM, Nate Finch nate.fi...@canonical.com wrote: On Thu, Aug 14, 2014 at 12:24 PM, Gustavo Niemeyer gustavo.nieme...@canonical.com wrote: Why support two things when you can support just one? Just to be clear, you really mean why support two existing and well

Re: getting rid of all-machines.log

2014-08-14 Thread Nate Finch
I didn't bring up 12 factor, it's irrelevant to my argument. I'm trying to make our product simpler and easier to maintain. That is all. If there's another cross-platform solution that we can use, I'd be happy to consider it. We have to change the code to support Windows. I'd rather the diff

Re: getting rid of all-machines.log

2014-08-14 Thread Gustavo Niemeyer
On Thu, Aug 14, 2014 at 3:14 PM, Nate Finch nate.fi...@canonical.com wrote: I didn't bring up 12 factor, it's irrelevant to my argument. Is there someone else sending messages under your name? On Thu, Aug 14, 2014 at 12:23 PM, Nate Finch nate.fi...@canonical.com wrote: The front page of

Re: getting rid of all-machines.log

2014-08-13 Thread David Cheney
Ian asked me to post my thoughts here. I am not in favour of applications rolling their own logs, I believe that applications should not know anything about their log output, they should just dump it all to stdout and another process should take care of shuttling the data, logging it, culling it,

Re: getting rid of all-machines.log

2014-08-13 Thread Ian Booth
Just to back up Dave's arguments - all sys admins I know would be a big -1 on Juju doing it's own log rolling. It's a recipe for lost log files, missing data etc. It's a mixing of responsibilities that should be handled separately. Just on the volume point Dave raised - we do log a lot but that's

Re: getting rid of all-machines.log

2014-08-08 Thread jaquilina
there is an rsyslog for windows but obviously it requires a license http://www.rsyslog.com/windows-agent/ On 2014-08-06 16:42, Nate Finch wrote: Fair enough, I forgot about debug-log.  Do we have an idea of how well aggregate logs from Windows machines?  rsyslog wont run there.   We might be

Re: getting rid of all-machines.log

2014-08-08 Thread Gabriel Samfira
The standard Go syslog package works great (with slight changes). As a bonus, it would be great to remove the dependency on rsyslog from all non bootstrap nodes if all we need it to do is tail the juju log and send it to an agregator :). As a native windows alternative, If we really do need to

Re: getting rid of all-machines.log

2014-08-08 Thread Nate Finch
I'm all for taking rsyslog out of the equation even on linux... what's nice about it is that, because we'd be writing log output separately to both the remote syslog and to the local file log, we wouldn't need to worry about log rotation of the local log screwing up what gets sent to the remote

Re: getting rid of all-machines.log

2014-08-08 Thread David Britton
On Fri, Aug 08, 2014 at 12:03:21PM -0400, Nate Finch wrote: [...] remote syslog and to the local file log, we wouldn't need to worry about log rotation of the local log screwing up what gets sent to the remote Do the standard rsyslog log rotation mechanisms not function well? On Windows, what

Re: getting rid of all-machines.log

2014-08-08 Thread Nate Finch
So, rsyslog rotation works fine on Linux, but we can't do that on Windows. If we have to do something different for Windows, I'd rather just do one thing which is cross platform compatible for all our OSes, and not have to support a different configuration for each OS. Doing it all

Re: getting rid of all-machines.log

2014-08-06 Thread Tim Penhey
On 06/08/14 16:11, Nate Finch wrote: all-machines.log seems both redundant and a ticking time bomb of disk space usage. Do we really need it? Can we drop it and maybe later schedule some time to use something like logstash that is both more featureful and is cross platform compatible (unlike

Re: getting rid of all-machines.log

2014-08-06 Thread Nate Finch
Fair enough, I forgot about debug-log. Do we have an idea of how we'll aggregate logs from Windows machines? rsyslog won't run there. We might be able to do a go-only solution by using the syslog package gabriel made a port of it that will run on Windows and supports TLS:

Re: getting rid of all-machines.log

2014-08-06 Thread John Meinel
That's probably a reasonable starting point. With the caveat that we know we don't really want to just use rsyslog forever. Just didn't get scheduled for this cycle. John =:- On Wed, Aug 6, 2014 at 4:42 PM, Nate Finch nate.fi...@canonical.com wrote: Fair enough, I forgot about debug-log. Do

Re: getting rid of all-machines.log

2014-08-06 Thread Aaron Bentley
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 14-08-06 10:11 AM, Nate Finch wrote: all-machines.log seems both redundant and a ticking time bomb of disk space usage. Do we really need it? Can we drop it and maybe later schedule some time to use something like logstash that is both more