Re: [openstack-dev] [openstack-ansible] To NTP, or not to NTP, that is the question

2015-09-22 Thread Jesse Pretorius 
On 22 September 2015 at 01:45, Major Hayden  wrote:

> On 09/21/2015 07:14 PM, Sergii Golovatiuk wrote:
> > Are any chance to configure chrony instead of ntpd? It acts more
> predictable on virtual environments.
>
> That's my plan, if I can find an upstream Ansible galaxy role to use. ;)
>

Now that we have the spec for independent role repositories approved [1],
an option is for us to register a role which implements chrony as the
network time mechanism if there isn't a suitable one already in Ansible
Galaxy. This role can be an optional add-on to the supported use-cases in
OpenStack-Ansible and can also be registered in Ansible Galaxy once it's
ready. :)

[1] https://review.openstack.org/213779
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-ansible] To NTP, or not to NTP, that is the question

2015-09-21 Thread Sergii Golovatiuk 
Hi,

Are any chance to configure chrony instead of ntpd? It acts more
predictable on virtual environments.

--
Best regards,
Sergii Golovatiuk,
Skype #golserge
IRC #holser

On Mon, Sep 21, 2015 at 4:11 PM, Jesse Pretorius 
wrote:

> On 18 September 2015 at 14:03, Major Hayden  wrote:
>
>> Hey there,
>>
>> I start working on a bug[1] last night about adding a managed NTP
>> configuration to openstack-ansible hosts.  My patch[2] gets chrony up and
>> running with configurable NTP servers, but I'm still struggling to meet the
>> "Proposal" section of the bug where the author has asked for non-infra
>> physical nodes to get their time from the infra nodes.  I can't figure out
>> how to make it work for AIO builds when one physical host is part of all of
>> the groups. ;)
>>
>> I'd argue that time synchronization is critical for a few areas:
>>
>>   1) Security/auditing when comparing logs
>>   2) Troubleshooting when comparing logs
>>   3) I've been told swift is time-sensitive
>>   4) MySQL/Galera don't like time drift
>>
>> However, there's a strong argument that this should be done by deployers,
>> and not via openstack-ansible.  I'm still *very* new to the project and I'd
>> like to hear some feedback from other folks.
>>
>> [1] https://bugs.launchpad.net/openstack-ansible/+bug/1413018
>> [2] https://review.openstack.org/#/c/225006/
>
>
> We have historically taken the stance of leaving something like this as a
> deployer concern - much like setting up host networking and setting host
> repositories. That said, there's value in opinionation based on best
> practices learned from hard-won lessons in the trenches.
>
> I'm somewhat on the fence with this. As-is I don't think the review should
> go in. That said, I'd be more open to an individual role being used to
> implement an appropriate network time configuration - whether that role be
> something that exists within Ansible Galaxy, or whether it's a new role in
> the current repository, or as its own repository in the OpenStack-Ansible
> 'big tent' as proposed in https://review.openstack.org/213779
>
> I do definitely think that there's value in preparing some documentation
> which will help prospective deployers understand how they can consume roles
> from Ansible Galaxy (or some role in an arbitrary repository) to solve
> common problems like this. The tooling is already in the OpenStack-Ansible
> repository, so all it needs is a guiding document which describes how to
> use it.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-ansible] To NTP, or not to NTP, that is the question

2015-09-21 Thread Major Hayden 
On 09/21/2015 07:14 PM, Sergii Golovatiuk wrote:
> Are any chance to configure chrony instead of ntpd? It acts more predictable 
> on virtual environments.

That's my plan, if I can find an upstream Ansible galaxy role to use. ;)

--
Major Hayden

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-ansible] To NTP, or not to NTP, that is the question

2015-09-21 Thread Jesse Pretorius 
On 18 September 2015 at 14:03, Major Hayden  wrote:

> Hey there,
>
> I start working on a bug[1] last night about adding a managed NTP
> configuration to openstack-ansible hosts.  My patch[2] gets chrony up and
> running with configurable NTP servers, but I'm still struggling to meet the
> "Proposal" section of the bug where the author has asked for non-infra
> physical nodes to get their time from the infra nodes.  I can't figure out
> how to make it work for AIO builds when one physical host is part of all of
> the groups. ;)
>
> I'd argue that time synchronization is critical for a few areas:
>
>   1) Security/auditing when comparing logs
>   2) Troubleshooting when comparing logs
>   3) I've been told swift is time-sensitive
>   4) MySQL/Galera don't like time drift
>
> However, there's a strong argument that this should be done by deployers,
> and not via openstack-ansible.  I'm still *very* new to the project and I'd
> like to hear some feedback from other folks.
>
> [1] https://bugs.launchpad.net/openstack-ansible/+bug/1413018
> [2] https://review.openstack.org/#/c/225006/


We have historically taken the stance of leaving something like this as a
deployer concern - much like setting up host networking and setting host
repositories. That said, there's value in opinionation based on best
practices learned from hard-won lessons in the trenches.

I'm somewhat on the fence with this. As-is I don't think the review should
go in. That said, I'd be more open to an individual role being used to
implement an appropriate network time configuration - whether that role be
something that exists within Ansible Galaxy, or whether it's a new role in
the current repository, or as its own repository in the OpenStack-Ansible
'big tent' as proposed in https://review.openstack.org/213779

I do definitely think that there's value in preparing some documentation
which will help prospective deployers understand how they can consume roles
from Ansible Galaxy (or some role in an arbitrary repository) to solve
common problems like this. The tooling is already in the OpenStack-Ansible
repository, so all it needs is a guiding document which describes how to
use it.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-ansible] To NTP, or not to NTP, that is the question

2015-09-19 Thread Jim Meyer 
> On Sep 18, 2015, at 9:38 AM, Jay Pipes  wrote:
> 
>> On 09/18/2015 11:04 AM, Ian Cordasco wrote:
>>> On 9/18/15, 08:03, "Major Hayden"  wrote:
>>> 
>>> Hey there,
>>> 
>>> I start working on a bug[1] last night about adding a managed NTP
>>> configuration to openstack-ansible hosts.  My patch[2] gets chrony up and
>>> running with configurable NTP servers, but I'm still struggling to meet
>>> the "Proposal" section of the bug where the author has asked for
>>> non-infra physical nodes to get their time from the infra nodes.  I can't
>>> figure out how to make it work for AIO builds when one physical host is
>>> part of all of the groups. ;)
>>> 
>>> I'd argue that time synchronization is critical for a few areas:
>>> 
>>>  1) Security/auditing when comparing logs
>>>  2) Troubleshooting when comparing logs
>>>  3) I've been told swift is time-sensitive
>>>  4) MySQL/Galera don't like time drift
>>> 
>>> However, there's a strong argument that this should be done by deployers,
>>> and not via openstack-ansible.  I'm still *very* new to the project and
>>> I'd like to hear some feedback from other folks.
>> 
>> Personally, I fall into the camp of "this is a deployer concern".
>> Specifically, there is already an ansible-galaxy role to enable NTP on
>> your deployment hosts (https://galaxy.ansible.com/list#/roles/464) which
>> *could* be expanded to do this very work that you're talking about. Using
>> specialized roles to achieve this (and contributing back to the larger
>> ansible community) seems like a bigger win than trying to reimplement some
>> of this in OSA instead of reusing other roles that already exist.
>> 
>> Compare it to a hypothetical situation where Keystone wrote its own
>> backing libraries to implement Fernet instead of using the cryptography
>> library. In that case there would be absolutely no argument that Keystone
>> should use cryptography (even if it uses cffi and has bindings to OpenSSL
>> which our infra team doesn't like and some deployers find difficult to
>> manage when using pure-python deployment tooling). Why should OSA be any
>> different from another OpenStack project?
> 
> Have to agree with Ian here. NTP, as Major wrote, is a critical piece of the 
> deployment puzzle, but I don't think it's necessary to put anything in OSA 
> specifically to configure NTP. As Ian wrote, better to contribute to upstream 
> ansible-galaxy playbooks/roles that do this well.

I have a nuanced agreement with this which borders on disagreement. 

An agreed-upon time tick is as crucial to a distributed system as oxygen is to 
a human. It's not only those components that care, it's the humans who have to 
understand and operate it. As such, an OpenStack cloud should come with a time 
source that all services listen to; even if it's wildly off from the real 
world, the value of all services sharing the same tick is immeasurable. For me, 
it's part of "batteries included."

I'd argue that we should pick a tool and configuration for this by default and 
allow others to change it. And, while I love Major*, I don't think the 
deployment tools are the right place for this.

--j

* and I do. Been too long, Major. We should fix that. =]
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-ansible] To NTP, or not to NTP, that is the question

2015-09-18 Thread Ian Cordasco 


On 9/18/15, 08:03, "Major Hayden"  wrote:

>Hey there,
>
>I start working on a bug[1] last night about adding a managed NTP
>configuration to openstack-ansible hosts.  My patch[2] gets chrony up and
>running with configurable NTP servers, but I'm still struggling to meet
>the "Proposal" section of the bug where the author has asked for
>non-infra physical nodes to get their time from the infra nodes.  I can't
>figure out how to make it work for AIO builds when one physical host is
>part of all of the groups. ;)
>
>I'd argue that time synchronization is critical for a few areas:
>
>  1) Security/auditing when comparing logs
>  2) Troubleshooting when comparing logs
>  3) I've been told swift is time-sensitive
>  4) MySQL/Galera don't like time drift
>
>However, there's a strong argument that this should be done by deployers,
>and not via openstack-ansible.  I'm still *very* new to the project and
>I'd like to hear some feedback from other folks.

Personally, I fall into the camp of "this is a deployer concern".
Specifically, there is already an ansible-galaxy role to enable NTP on
your deployment hosts (https://galaxy.ansible.com/list#/roles/464) which
*could* be expanded to do this very work that you're talking about. Using
specialized roles to achieve this (and contributing back to the larger
ansible community) seems like a bigger win than trying to reimplement some
of this in OSA instead of reusing other roles that already exist.

Compare it to a hypothetical situation where Keystone wrote its own
backing libraries to implement Fernet instead of using the cryptography
library. In that case there would be absolutely no argument that Keystone
should use cryptography (even if it uses cffi and has bindings to OpenSSL
which our infra team doesn't like and some deployers find difficult to
manage when using pure-python deployment tooling). Why should OSA be any
different from another OpenStack project?

Cheers,
Ian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [openstack-ansible] To NTP, or not to NTP, that is the question

2015-09-18 Thread Major Hayden 
Hey there,

I start working on a bug[1] last night about adding a managed NTP configuration 
to openstack-ansible hosts.  My patch[2] gets chrony up and running with 
configurable NTP servers, but I'm still struggling to meet the "Proposal" 
section of the bug where the author has asked for non-infra physical nodes to 
get their time from the infra nodes.  I can't figure out how to make it work 
for AIO builds when one physical host is part of all of the groups. ;)

I'd argue that time synchronization is critical for a few areas:

  1) Security/auditing when comparing logs
  2) Troubleshooting when comparing logs
  3) I've been told swift is time-sensitive
  4) MySQL/Galera don't like time drift

However, there's a strong argument that this should be done by deployers, and 
not via openstack-ansible.  I'm still *very* new to the project and I'd like to 
hear some feedback from other folks.

[1] https://bugs.launchpad.net/openstack-ansible/+bug/1413018
[2] https://review.openstack.org/#/c/225006/

--
Major Hayden

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-ansible] To NTP, or not to NTP, that is the question

2015-09-18 Thread Matthew Mosesohn 
Major,

in Fuel, we've dealt with this problem for a long time in its varying
degrees of unpleasantness. Some virtualization platforms, such as
VirtualBox, are very prone to time drift. Hardware nodes, thankfully, don't
suffer so badly.

Time sync is very important for RabbitMQ, Corosync, and Ceph, in addition
to those items you mentioned above. I haven't seen swift itself break due
to time issues, but you may be right.

The most ideal situation is to point all hosts to public NTP pool servers.
Barring that, elect 1 host to base its time by its hardware clock, and then
direct all other hosts to sync time against that one host. This has major
issues when you're doing virtual deployments with snapshot/revert and
experiencing major time skew, so you may need extra VM management scripts
to manually sync time again after revert.


Best Regards,
Matthew Mosesohn

On Fri, Sep 18, 2015 at 4:03 PM, Major Hayden  wrote:

> Hey there,
>
> I start working on a bug[1] last night about adding a managed NTP
> configuration to openstack-ansible hosts.  My patch[2] gets chrony up and
> running with configurable NTP servers, but I'm still struggling to meet the
> "Proposal" section of the bug where the author has asked for non-infra
> physical nodes to get their time from the infra nodes.  I can't figure out
> how to make it work for AIO builds when one physical host is part of all of
> the groups. ;)
>
> I'd argue that time synchronization is critical for a few areas:
>
>   1) Security/auditing when comparing logs
>   2) Troubleshooting when comparing logs
>   3) I've been told swift is time-sensitive
>   4) MySQL/Galera don't like time drift
>
> However, there's a strong argument that this should be done by deployers,
> and not via openstack-ansible.  I'm still *very* new to the project and I'd
> like to hear some feedback from other folks.
>
> [1] https://bugs.launchpad.net/openstack-ansible/+bug/1413018
> [2] https://review.openstack.org/#/c/225006/
>
> --
> Major Hayden
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-ansible] To NTP, or not to NTP, that is the question

2015-09-18 Thread Jay Pipes 

On 09/18/2015 11:04 AM, Ian Cordasco wrote:

On 9/18/15, 08:03, "Major Hayden"  wrote:


Hey there,

I start working on a bug[1] last night about adding a managed NTP
configuration to openstack-ansible hosts.  My patch[2] gets chrony up and
running with configurable NTP servers, but I'm still struggling to meet
the "Proposal" section of the bug where the author has asked for
non-infra physical nodes to get their time from the infra nodes.  I can't
figure out how to make it work for AIO builds when one physical host is
part of all of the groups. ;)

I'd argue that time synchronization is critical for a few areas:

  1) Security/auditing when comparing logs
  2) Troubleshooting when comparing logs
  3) I've been told swift is time-sensitive
  4) MySQL/Galera don't like time drift

However, there's a strong argument that this should be done by deployers,
and not via openstack-ansible.  I'm still *very* new to the project and
I'd like to hear some feedback from other folks.


Personally, I fall into the camp of "this is a deployer concern".
Specifically, there is already an ansible-galaxy role to enable NTP on
your deployment hosts (https://galaxy.ansible.com/list#/roles/464) which
*could* be expanded to do this very work that you're talking about. Using
specialized roles to achieve this (and contributing back to the larger
ansible community) seems like a bigger win than trying to reimplement some
of this in OSA instead of reusing other roles that already exist.

Compare it to a hypothetical situation where Keystone wrote its own
backing libraries to implement Fernet instead of using the cryptography
library. In that case there would be absolutely no argument that Keystone
should use cryptography (even if it uses cffi and has bindings to OpenSSL
which our infra team doesn't like and some deployers find difficult to
manage when using pure-python deployment tooling). Why should OSA be any
different from another OpenStack project?


Have to agree with Ian here. NTP, as Major wrote, is a critical piece of 
the deployment puzzle, but I don't think it's necessary to put anything 
in OSA specifically to configure NTP. As Ian wrote, better to contribute 
to upstream ansible-galaxy playbooks/roles that do this well.


Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-ansible] To NTP, or not to NTP, that is the question

2015-09-18 Thread Fox, Kevin M 
My $0.02
Support configuring ntp. Have a flag that can turn that piece off. Default it 
on. Profit. :)

Thanks,
Kevin

From: Major Hayden [ma...@mhtx.net]
Sent: Friday, September 18, 2015 6:03 AM
To: openstack-dev@lists.openstack.org
Subject: [openstack-dev] [openstack-ansible] To NTP, or not to NTP, that is 
the question

Hey there,

I start working on a bug[1] last night about adding a managed NTP configuration 
to openstack-ansible hosts.  My patch[2] gets chrony up and running with 
configurable NTP servers, but I'm still struggling to meet the "Proposal" 
section of the bug where the author has asked for non-infra physical nodes to 
get their time from the infra nodes.  I can't figure out how to make it work 
for AIO builds when one physical host is part of all of the groups. ;)

I'd argue that time synchronization is critical for a few areas:

  1) Security/auditing when comparing logs
  2) Troubleshooting when comparing logs
  3) I've been told swift is time-sensitive
  4) MySQL/Galera don't like time drift

However, there's a strong argument that this should be done by deployers, and 
not via openstack-ansible.  I'm still *very* new to the project and I'd like to 
hear some feedback from other folks.

[1] https://bugs.launchpad.net/openstack-ansible/+bug/1413018
[2] https://review.openstack.org/#/c/225006/

--
Major Hayden

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev