On 25 July 2014 08:01, Sean Dague <s...@dague.net> wrote: >> I'd like us to think about whether they is anything we can do to make >> life easier in these kind of hard debugging scenarios where the regular >> logs are not sufficient. > > Agreed. Honestly, though we do also need to figure out first fail > detection on our logs as well. Because realistically if we can't debug > failures from those, then I really don't understand how we're ever going > to expect large users to.
I'm so glad you said that :). In conversations with our users, and existing large deployers of Openstack, one thing has come through very consistently: our default logs are insufficient. We had an extensive discussion about this in the TripleO mid-cycle meetup, and I think we reached broad consensus on the following: - the defaults should be what folk are running in production - we don't want to lead on changing defaults - its a big enough thing we want to drive the discussion but not workaround it by changing our defaults - large clouds are *today* running debug (with a few tweaks to remove the most egregious log spammers and known security issues [like dumping tokens into logs] - AFAICT productised clouds (push-button deploy etc) are running something very similar - we would love it if developers *also* saw what users will see by default, since that will tend to both stop things getting to spammy, and too sparse. So - I know thats brief - what we'd like to do is to poll a slightly wider set of deployers - e.g. via a spec, perhaps some help from Tom with the users and ops groups - and get a baseline of things that there is consensus on and things that aren't, and then just change the defaults to match. Further, to achieve the 'developers see the same thing as users' bit, we'd like to make devstack do what TripleO does - use defaults for logging levels, particularly in the gate. Its totally true that we have a good policy about logging and we're changing things to fit it but thats the long term play: short term, making the default meet our deployments seems realtively easy and immensely sane. -Rob -- Robert Collins <rbtcoll...@hp.com> Distinguished Technologist HP Converged Cloud _______________________________________________ OpenStack-dev mailing list OpenStackemail@example.com http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev