Hi Charles, Just focusing on the logstash-agent charm, I really like how you managed additional log files with a server centric view. Thanks for that. Now adding new "log management workloads" just became much easier than it used to be. And it also became much more "cloud opiniated" which I like.
One thing I would eventually add is a way to import inputs from an external repository without having to update the charm itself. There are about 40 inputs available for LogStash, divided in 2 groups: * Those that are remote collection (logstash agents collects data from a remote node), which therefore require a specific relation or the agent to be in fact a broker (so I would host that only on an indexer node or create a new class of logstash agent is are not subordinate) ==> Those can be handled later or community based. They are collectd, drupal_dblog, elasticsearch, ganglia... and the rest of the remote logstash from http://logstash.net/docs/1.4.2/. I would see that as a relation exposing a name + port (+gist?), which would basically make logstash create a new input_<name>_<port>.conf file with content of the gist or a default configuration. * Those that only talk to a local instance, thus match the subordinate mechanism. For those, offering a way to download a remote configuration file would really be helpful. My personal view on this is that people would probably maintain a git repo with many configuration files (one per input probably, maybe one per class) hence a way to manage this would be to offer them 2 options: one for the git repo, one for a cron refresh of the configuration. In essence, this would trigger the download of the git repo, copy/link files to the logstash config folder and restart the agent. I think this would make the charm even more awesome :) More feedback as I test this. Thanks for this, it opens a new field to move to production with Juju! Sam On Mon, Oct 20, 2014 at 10:57 PM, Charles Butler < [email protected]> wrote: > I just pushed up a new demonstration bundle which leverages my changes to > the trusty logstash branch for routing all logs through the redis interface > of the indexer (which is apparently faster than just scaling elasticsearch > and just directing the inputs there...) > > I'd love it if someone could pull this branch and give it a go with > feedback: > > https://code.launchpad.net/~lazypower/charms/bundles/big-data-logger/bundle > > branch that, and deploy with: > > juju deployer -c bundles.yaml > juju expose kibana > > you should be able to see events from every service in the cluster at: > > http://{ip-of-kibana}/#/dashboard/file/logstash.json > > > Thanks in advance for any feedback! > > All the best, > > Charles > > -- > Juju mailing list > [email protected] > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju > > -- Samuel Cozannet Cloud, Big Data and IoT Strategy Team Strategic Program Manager Changing the Future of Cloud Ubuntu <http://ubuntu.com> / Canonical <http://canonical.com> UK LTD [email protected] +33 616 702 389
-- Juju mailing list [email protected] Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
