jhg03a commented on a change in pull request #3585: URL: https://github.com/apache/trafficcontrol/pull/3585#discussion_r494570610
########## File path: docs/source/admin/ansible-labs/ansible_labs.rst ########## @@ -0,0 +1,172 @@ +.. +.. +.. Licensed under the Apache License, Version 2.0 (the "License"); +.. you may not use this file except in compliance with the License. +.. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed under the License is distributed on an "AS IS" BASIS, +.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +.. See the License for the specific language governing permissions and +.. limitations under the License. +.. + +.. _ansiblelab: + +************************** +Ansible-based Lab Creation +************************** + +The scope of the Ansible work presented is a set of generic roles an implementor could use to expose the common installation and configuration tasks of each component. +Additionally, it provides some suggestions and sample scaffolding on how to divide the full end-to-end lab construction. + +.. topic:: Why Ansible? + + There are many excellent tools to facilitate application installation and configuration. + Ansible is a leading open-source tool in this marketspace backed by major corporate sponsorship and adoption. + Most importantly it facilitates the abstractions desired without creating technological conflicts with existing infrastructure management solutions. + +.. topic:: What about Security? + + Each organization should review the instructions being performed in each Ansible playbook to determine if they satisfy their security requirements. + Additionally, each implementor should select and implement their secret store of choice such as the built-in ``ansible-vault`` or a more advanced secret-as-a-service solution for any sensitive variables. + +Lab Implementation Concepts +=========================== + +.. figure:: ATC.lab.layers.svg + :scale: 100 % + :align: center + :figclass: align-center + +The basic idea is to separate responsibilities to allow each implementation to use the tools/technologies that are already in use within their organizations. + +Provisioning Layer +------------------ + +The provisioning layer deals with the lowest levels of compute/network/storage/load balancer resources. +Its objective is to bring systems from nothingness to a functional operating system (at least minimally). +Additionally, it is responsible for setting up proper DNS for each system, CDN components, and DNS NS record delegations. + +Since DNS is a part of this layer, this unfortunately necessitates a small number of CDN concepts being present in the provisioning configuration. +It is expected that upon completion of this layer, a compatible Ansible inventory file is generated and placed in the lab's Ansible inventory directory. + +The Provisioning Layer Output +""""""""""""""""""""""""""""" + +Ansible supports inventory files in several formats such as JSON, YAML, INI, or TOML. +An example output is located at :atc-file:`infrastructure/ansible/sample.lab/inventory/provisioning.inventory`. + +When creating systems, each will probably be designated for some particular component in the CDN. +Ansible groups and hostvars specifying the component name must be provided and will be mapped to server types later on in the dataset loader. + +Like the systems being allocated to a particular component name, they must also be assigned to a particular DNS NS Delegation name where relevant or the ``ALL`` cdn as needed by the component. +The NS Delegation name is what you would find in DNS for Traffic Router to be authoritative for. In this example it's ``MKGA`` rather than the CDN Name which is ``Kabletown2.0``. + +It is not recommended that an ::term::`origin` be used by more than one ::term::`delivery service` or it could lead to inconsistent behavior and conflated log data at the ::term::`Mid-tier`. +As a workaround to this it is better for a lab to create DNS CNAME for each ::term::`delivery service` pointing at a particular ::term::`origin` and return that set of names as a CSV hostvar ds_names on each ::term::`origin`. +These names will later be translated to additional inventory hosts used only for the creation of server objects in Traffic Ops and assignment to ::term::`delivery services`. Review comment: Addressed with commit ########## File path: docs/source/admin/traffic_ops/installation.rst ########## @@ -0,0 +1,232 @@ +.. Review comment: Addressed with commit ########## File path: infrastructure/ansible/dynamic.inventory/README.md ########## @@ -0,0 +1,44 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> +# Ansible Dynamic Inventory + +This python script uses the Traffic Ops Python Native Client to expose much of the TO dataset as an ansible inventory on demand as Ansible patterns. + +## Requirements +You will need to ensure the Traffic Ops Python Native Client is available to the python env shared by Ansible (https://github.com/apache/trafficcontrol/tree/master/traffic_control/clients/python). + +Due to limitations in the way parameters are passed in Ansible Dynamic Inventory scripts, the following environment variables must be defined: +```bash session +export TO_USER=<my.to.username> +export TO_PASSWORD=<my.to.password> +export TO_URL=to.kabletown.invalid Review comment: Addressed with commit ########## File path: infrastructure/ansible/dynamic.inventory/TO.py ########## @@ -0,0 +1,243 @@ +#!/usr/bin/env python3 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import json +import argparse +import logging Review comment: Addressed with commit ########## File path: infrastructure/ansible/dynamic.inventory/TO.py ########## @@ -0,0 +1,243 @@ +#!/usr/bin/env python3 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import json +import argparse +import logging +import os +import collections +from trafficops.tosession import TOSession + +def empty_inventory(): + """Generate a valid empty inventory""" + return {'_meta': {'hostvars': {}}} + + +class AnsibleInventory(): + """Wrapper class for needed methods""" + + def __init__(self, user, password, url, verify_cert): + """Init base members""" + self.to_user = user + self.to_pass = password + self.to_url = url + self.verify_cert = verify_cert + + @staticmethod + def populate_server_profile_vars(api, profile_id): + """Generate the server profile variables once as we see it""" + server_vars = {} + server_vars['hosts'] = [] + server_vars['vars'] = {} + profile = api.get_profiles(id=profile_id)[0] + server_vars['vars']['server_profile_description'] = profile[0]['description'] + server_vars['vars']['server_profile_type'] = profile[0]['type'] + server_vars['vars']['server_profile_routingDisabled'] = profile[0]['routingDisabled'] + server_vars['vars']['server_profile_parameters'] = [] + params = api.get_parameters_by_profile_id(id=profile_id)[0] + for param in params: + tmp_param = { + 'name': param['name'], + 'value': param['value'], + 'configFile': param['configFile']} + server_vars['vars']['server_profile_parameters'].append(tmp_param) + return server_vars + + @staticmethod + def populate_cachegroups(api, cachegroup_id): + """Generate the values for cachegroups once on first sight""" + var_data = {} + cgdata = collections.namedtuple('Cgdata', ['cgvars', + 'primary_parent_group_name', + 'secondary_parent_group_name']) + var_data['hosts'] = [] + var_data['vars'] = {} + cachegroup = api.get_cachegroups(id=cachegroup_id)[0] + var_data['vars']['cachegroup_name'] = cachegroup[0]['name'] + var_data['vars']['cachegroup_shortName'] = cachegroup[0]['shortName'] + var_data['vars']['cachegroup_parentCachegroupName'] = \ + cachegroup[0]['parentCachegroupName'] + var_data['vars']['cachegroup_secondaryParentCachegroupName'] = \ + cachegroup[0]['secondaryParentCachegroupName'] + var_data['vars']['cachegroup_typeName'] = cachegroup[0]['typeName'] + if cachegroup[0]['parentCachegroupName'] is None: + flat_parent_cg = "parentCachegroup|None" + else: + flat_parent_cg = "parentCachegroup|" + \ + cachegroup[0]['parentCachegroupName'] + + if cachegroup[0]['secondaryParentCachegroupName'] is None: + flat_second_parent_cg = "secondaryParentCachegroup|None" + else: + flat_second_parent_cg = "secondaryParentCachegroup|" + \ + cachegroup[0]['secondaryParentCachegroupName'] + out = cgdata(cgvars=var_data, + primary_parent_group_name=flat_parent_cg, + secondary_parent_group_name=flat_second_parent_cg) + return out + + def generate_inventory_list(self, target_to): # pylint: disable=too-many-statements Review comment: Addressed with commit ########## File path: infrastructure/ansible/dynamic.inventory/TO.py ########## @@ -0,0 +1,243 @@ +#!/usr/bin/env python3 Review comment: Addressed with commit ########## File path: infrastructure/ansible/dynamic.inventory/TO.py ########## @@ -0,0 +1,243 @@ +#!/usr/bin/env python3 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import json +import argparse +import logging +import os +import collections +from trafficops.tosession import TOSession + +def empty_inventory(): + """Generate a valid empty inventory""" + return {'_meta': {'hostvars': {}}} + + +class AnsibleInventory(): + """Wrapper class for needed methods""" + + def __init__(self, user, password, url, verify_cert): + """Init base members""" + self.to_user = user + self.to_pass = password + self.to_url = url + self.verify_cert = verify_cert + + @staticmethod + def populate_server_profile_vars(api, profile_id): + """Generate the server profile variables once as we see it""" + server_vars = {} + server_vars['hosts'] = [] + server_vars['vars'] = {} + profile = api.get_profiles(id=profile_id)[0] + server_vars['vars']['server_profile_description'] = profile[0]['description'] + server_vars['vars']['server_profile_type'] = profile[0]['type'] + server_vars['vars']['server_profile_routingDisabled'] = profile[0]['routingDisabled'] + server_vars['vars']['server_profile_parameters'] = [] + params = api.get_parameters_by_profile_id(id=profile_id)[0] + for param in params: + tmp_param = { + 'name': param['name'], + 'value': param['value'], + 'configFile': param['configFile']} + server_vars['vars']['server_profile_parameters'].append(tmp_param) + return server_vars + + @staticmethod + def populate_cachegroups(api, cachegroup_id): + """Generate the values for cachegroups once on first sight""" + var_data = {} + cgdata = collections.namedtuple('Cgdata', ['cgvars', + 'primary_parent_group_name', + 'secondary_parent_group_name']) + var_data['hosts'] = [] + var_data['vars'] = {} + cachegroup = api.get_cachegroups(id=cachegroup_id)[0] + var_data['vars']['cachegroup_name'] = cachegroup[0]['name'] + var_data['vars']['cachegroup_shortName'] = cachegroup[0]['shortName'] + var_data['vars']['cachegroup_parentCachegroupName'] = \ + cachegroup[0]['parentCachegroupName'] + var_data['vars']['cachegroup_secondaryParentCachegroupName'] = \ + cachegroup[0]['secondaryParentCachegroupName'] + var_data['vars']['cachegroup_typeName'] = cachegroup[0]['typeName'] + if cachegroup[0]['parentCachegroupName'] is None: + flat_parent_cg = "parentCachegroup|None" + else: + flat_parent_cg = "parentCachegroup|" + \ + cachegroup[0]['parentCachegroupName'] + + if cachegroup[0]['secondaryParentCachegroupName'] is None: + flat_second_parent_cg = "secondaryParentCachegroup|None" + else: + flat_second_parent_cg = "secondaryParentCachegroup|" + \ + cachegroup[0]['secondaryParentCachegroupName'] + out = cgdata(cgvars=var_data, + primary_parent_group_name=flat_parent_cg, + secondary_parent_group_name=flat_second_parent_cg) + return out + + def generate_inventory_list(self, target_to): # pylint: disable=too-many-statements + """Generate the inventory list for the specified TrafficOps instance""" + with TOSession(self.to_url, verify_cert=self.verify_cert) as traffic_ops_api: + traffic_ops_api.login(self.to_user, self.to_pass) + servers = traffic_ops_api.get_servers()[0] + out = {} + out['_meta'] = {} + out['_meta']['hostvars'] = {} + out[target_to] = {} + out[target_to]['hosts'] = [] + out["ungrouped"] = {} + out['ungrouped']['hosts'] = [] + out['cachegroup'] = {} + out['cachegroup']['children'] = [] + out['server_type'] = {} + out['server_type']['children'] = [] + out['server_cdnName'] = {} + out['server_cdnName']['children'] = [] + out['server_profile'] = {} + out['server_profile']['children'] = [] + out['server_status'] = {} + out['server_status']['children'] = [] + for server in servers: + fqdn = server['hostName'] + '.' + server['domainName'] + out["ungrouped"]['hosts'].append(fqdn) + out[target_to]['hosts'].append(fqdn) + out['_meta']['hostvars'][fqdn] = {} + out['_meta']['hostvars'][fqdn]['server_toFQDN'] = target_to + out['_meta']['hostvars'][fqdn]['server_cachegroup'] = server['cachegroup'] + out['_meta']['hostvars'][fqdn]['server_cdnName'] = server['cdnName'] + out['_meta']['hostvars'][fqdn]['server_id'] = server['id'] + out['_meta']['hostvars'][fqdn]['server_ipAddress'] = server['ipAddress'] + out['_meta']['hostvars'][fqdn]['server_ip6Address'] = server['ip6Address'] + out['_meta']['hostvars'][fqdn]['server_offlineReason'] = server['offlineReason'] + out['_meta']['hostvars'][fqdn]['server_physLocation'] = server['physLocation'] + out['_meta']['hostvars'][fqdn]['server_profile'] = server['profile'] + out['_meta']['hostvars'][fqdn]['server_profileDesc'] = server['profileDesc'] + out['_meta']['hostvars'][fqdn]['server_status'] = server['status'] + out['_meta']['hostvars'][fqdn]['server_type'] = server['type'] + flat_server_profile = "server_profile|" + server['profile'] + flat_cachegroup = "cachegroup|" + server['cachegroup'] + flat_server_type = "server_type|" + server['type'] + flat_server_cdn_name = "server_cdnName|" + server['cdnName'] + flat_server_status = "server_status|" + server['status'] + if flat_server_profile not in out: + out['server_profile']['children'].append(flat_server_profile) + out[flat_server_profile] = self.populate_server_profile_vars( + traffic_ops_api, + server['profileId']) + out[flat_server_profile]['hosts'].append(fqdn) + if flat_cachegroup not in out: + out['cachegroup']['children'].append(flat_cachegroup) + cgdata = self.populate_cachegroups( + traffic_ops_api, + server['cachegroupId']) + out[flat_cachegroup] = cgdata.cgvars + flat_parent_cg = cgdata.primary_parent_group_name + flat_second_parent_cg = cgdata.secondary_parent_group_name + if flat_parent_cg not in out: + out[flat_parent_cg] = {} + out[flat_parent_cg]['children'] = [] + if flat_second_parent_cg not in out: + out[flat_second_parent_cg] = {} + out[flat_second_parent_cg]['children'] = [] + out[flat_parent_cg]['children'].append(flat_cachegroup) + out[flat_second_parent_cg]['children'].append(flat_cachegroup) + out[flat_cachegroup]['hosts'].append(fqdn) + if flat_server_type not in out: + out['server_type']['children'].append(flat_server_type) + out[flat_server_type] = {} + out[flat_server_type]['hosts'] = [] + out[flat_server_type]['hosts'].append(fqdn) + if flat_server_cdn_name not in out: + out['server_cdnName']['children'].append(flat_server_cdn_name) + out[flat_server_cdn_name] = {} + out[flat_server_cdn_name]['hosts'] = [] + out[flat_server_cdn_name]['hosts'].append(fqdn) + if flat_server_status not in out: + out['server_status']['children'].append(flat_server_status) + out[flat_server_status] = {} + out[flat_server_status]['hosts'] = [] + out[flat_server_status]['hosts'].append(fqdn) + return out + + def to_inventory(self): + return self.generate_inventory_list(self.to_url) + +# +# Thanks to Maxim for the snipit on handling bool parameters. +# https://stackoverflow.com/questions/15008758/parsing-boolean-values-with-argparse +# + + +def str2bool(v): Review comment: Addressed with commit ########## File path: infrastructure/ansible/dynamic.inventory/TO.py ########## @@ -0,0 +1,243 @@ +#!/usr/bin/env python3 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import json +import argparse +import logging +import os +import collections +from trafficops.tosession import TOSession + +def empty_inventory(): + """Generate a valid empty inventory""" + return {'_meta': {'hostvars': {}}} + + +class AnsibleInventory(): + """Wrapper class for needed methods""" + + def __init__(self, user, password, url, verify_cert): + """Init base members""" + self.to_user = user + self.to_pass = password + self.to_url = url + self.verify_cert = verify_cert + + @staticmethod + def populate_server_profile_vars(api, profile_id): + """Generate the server profile variables once as we see it""" + server_vars = {} + server_vars['hosts'] = [] + server_vars['vars'] = {} + profile = api.get_profiles(id=profile_id)[0] + server_vars['vars']['server_profile_description'] = profile[0]['description'] + server_vars['vars']['server_profile_type'] = profile[0]['type'] + server_vars['vars']['server_profile_routingDisabled'] = profile[0]['routingDisabled'] + server_vars['vars']['server_profile_parameters'] = [] + params = api.get_parameters_by_profile_id(id=profile_id)[0] + for param in params: + tmp_param = { + 'name': param['name'], + 'value': param['value'], + 'configFile': param['configFile']} + server_vars['vars']['server_profile_parameters'].append(tmp_param) + return server_vars + + @staticmethod + def populate_cachegroups(api, cachegroup_id): + """Generate the values for cachegroups once on first sight""" + var_data = {} + cgdata = collections.namedtuple('Cgdata', ['cgvars', + 'primary_parent_group_name', + 'secondary_parent_group_name']) + var_data['hosts'] = [] + var_data['vars'] = {} + cachegroup = api.get_cachegroups(id=cachegroup_id)[0] + var_data['vars']['cachegroup_name'] = cachegroup[0]['name'] + var_data['vars']['cachegroup_shortName'] = cachegroup[0]['shortName'] + var_data['vars']['cachegroup_parentCachegroupName'] = \ + cachegroup[0]['parentCachegroupName'] + var_data['vars']['cachegroup_secondaryParentCachegroupName'] = \ + cachegroup[0]['secondaryParentCachegroupName'] + var_data['vars']['cachegroup_typeName'] = cachegroup[0]['typeName'] + if cachegroup[0]['parentCachegroupName'] is None: + flat_parent_cg = "parentCachegroup|None" + else: + flat_parent_cg = "parentCachegroup|" + \ + cachegroup[0]['parentCachegroupName'] + + if cachegroup[0]['secondaryParentCachegroupName'] is None: + flat_second_parent_cg = "secondaryParentCachegroup|None" + else: + flat_second_parent_cg = "secondaryParentCachegroup|" + \ + cachegroup[0]['secondaryParentCachegroupName'] + out = cgdata(cgvars=var_data, + primary_parent_group_name=flat_parent_cg, + secondary_parent_group_name=flat_second_parent_cg) + return out + + def generate_inventory_list(self, target_to): # pylint: disable=too-many-statements + """Generate the inventory list for the specified TrafficOps instance""" + with TOSession(self.to_url, verify_cert=self.verify_cert) as traffic_ops_api: + traffic_ops_api.login(self.to_user, self.to_pass) + servers = traffic_ops_api.get_servers()[0] + out = {} + out['_meta'] = {} + out['_meta']['hostvars'] = {} + out[target_to] = {} + out[target_to]['hosts'] = [] + out["ungrouped"] = {} + out['ungrouped']['hosts'] = [] + out['cachegroup'] = {} + out['cachegroup']['children'] = [] + out['server_type'] = {} + out['server_type']['children'] = [] + out['server_cdnName'] = {} + out['server_cdnName']['children'] = [] + out['server_profile'] = {} + out['server_profile']['children'] = [] + out['server_status'] = {} + out['server_status']['children'] = [] + for server in servers: + fqdn = server['hostName'] + '.' + server['domainName'] + out["ungrouped"]['hosts'].append(fqdn) + out[target_to]['hosts'].append(fqdn) + out['_meta']['hostvars'][fqdn] = {} + out['_meta']['hostvars'][fqdn]['server_toFQDN'] = target_to + out['_meta']['hostvars'][fqdn]['server_cachegroup'] = server['cachegroup'] + out['_meta']['hostvars'][fqdn]['server_cdnName'] = server['cdnName'] + out['_meta']['hostvars'][fqdn]['server_id'] = server['id'] + out['_meta']['hostvars'][fqdn]['server_ipAddress'] = server['ipAddress'] + out['_meta']['hostvars'][fqdn]['server_ip6Address'] = server['ip6Address'] + out['_meta']['hostvars'][fqdn]['server_offlineReason'] = server['offlineReason'] + out['_meta']['hostvars'][fqdn]['server_physLocation'] = server['physLocation'] + out['_meta']['hostvars'][fqdn]['server_profile'] = server['profile'] + out['_meta']['hostvars'][fqdn]['server_profileDesc'] = server['profileDesc'] + out['_meta']['hostvars'][fqdn]['server_status'] = server['status'] + out['_meta']['hostvars'][fqdn]['server_type'] = server['type'] + flat_server_profile = "server_profile|" + server['profile'] + flat_cachegroup = "cachegroup|" + server['cachegroup'] + flat_server_type = "server_type|" + server['type'] + flat_server_cdn_name = "server_cdnName|" + server['cdnName'] + flat_server_status = "server_status|" + server['status'] + if flat_server_profile not in out: + out['server_profile']['children'].append(flat_server_profile) + out[flat_server_profile] = self.populate_server_profile_vars( + traffic_ops_api, + server['profileId']) + out[flat_server_profile]['hosts'].append(fqdn) + if flat_cachegroup not in out: + out['cachegroup']['children'].append(flat_cachegroup) + cgdata = self.populate_cachegroups( + traffic_ops_api, + server['cachegroupId']) + out[flat_cachegroup] = cgdata.cgvars + flat_parent_cg = cgdata.primary_parent_group_name + flat_second_parent_cg = cgdata.secondary_parent_group_name + if flat_parent_cg not in out: + out[flat_parent_cg] = {} + out[flat_parent_cg]['children'] = [] + if flat_second_parent_cg not in out: + out[flat_second_parent_cg] = {} + out[flat_second_parent_cg]['children'] = [] + out[flat_parent_cg]['children'].append(flat_cachegroup) + out[flat_second_parent_cg]['children'].append(flat_cachegroup) + out[flat_cachegroup]['hosts'].append(fqdn) + if flat_server_type not in out: + out['server_type']['children'].append(flat_server_type) + out[flat_server_type] = {} + out[flat_server_type]['hosts'] = [] + out[flat_server_type]['hosts'].append(fqdn) + if flat_server_cdn_name not in out: + out['server_cdnName']['children'].append(flat_server_cdn_name) + out[flat_server_cdn_name] = {} + out[flat_server_cdn_name]['hosts'] = [] + out[flat_server_cdn_name]['hosts'].append(fqdn) + if flat_server_status not in out: + out['server_status']['children'].append(flat_server_status) + out[flat_server_status] = {} + out[flat_server_status]['hosts'] = [] + out[flat_server_status]['hosts'].append(fqdn) + return out + + def to_inventory(self): + return self.generate_inventory_list(self.to_url) + +# +# Thanks to Maxim for the snipit on handling bool parameters. +# https://stackoverflow.com/questions/15008758/parsing-boolean-values-with-argparse +# + + +def str2bool(v): + if isinstance(v, bool): + return v + if v.lower() in ('yes', 'true', 't', 'y', '1'): + return True + elif v.lower() in ('no', 'false', 'f', 'n', '0'): Review comment: Addressed with commit ########## File path: infrastructure/ansible/roles/dataset_loader/files/selection.set.py ########## @@ -0,0 +1,49 @@ +#!/usr/bin/python Review comment: Addressed with commit ########## File path: infrastructure/ansible/roles/dataset_loader/files/selection.set.py ########## @@ -0,0 +1,49 @@ +#!/usr/bin/python + +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# This script is used to provide a round-robin merging of two lists + +import sys +import json + +if len(sys.argv) < 3 or len(sys.argv) > 4: + print "{}" Review comment: Addressed with commit ########## File path: infrastructure/ansible/roles/dataset_loader/files/selection.set.py ########## @@ -0,0 +1,49 @@ +#!/usr/bin/python + +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# This script is used to provide a round-robin merging of two lists + +import sys +import json + +if len(sys.argv) < 3 or len(sys.argv) > 4: + print "{}" + sys.exit(0) + +cdn_csv_list = sys.argv[1].split(',') +fqdn_csv_list = sys.argv[2].split(',') +option = '' +if len(sys.argv) == 4: + option = sys.argv[3] +cdn_csv_list.sort() +fqdn_csv_list.sort() + +step_size = len(cdn_csv_list) +out_list_normal = {} +for i, val in enumerate(cdn_csv_list): + sublist = fqdn_csv_list[i:] + out_list_normal[val] = ','.join(sublist[::step_size]) + +out_list_denormal = {} +for val, csvlist in out_list_normal.items(): + for i in csvlist.split(','): + if i != "": + out_list_denormal[i] = val + +if option == 'denormalize': + print json.dumps(out_list_denormal) Review comment: Addressed with commit ########## File path: infrastructure/ansible/roles/dataset_loader/files/selection.set.py ########## @@ -0,0 +1,49 @@ +#!/usr/bin/python + +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# This script is used to provide a round-robin merging of two lists + +import sys +import json + +if len(sys.argv) < 3 or len(sys.argv) > 4: + print "{}" + sys.exit(0) + +cdn_csv_list = sys.argv[1].split(',') +fqdn_csv_list = sys.argv[2].split(',') +option = '' +if len(sys.argv) == 4: + option = sys.argv[3] +cdn_csv_list.sort() +fqdn_csv_list.sort() + +step_size = len(cdn_csv_list) +out_list_normal = {} +for i, val in enumerate(cdn_csv_list): + sublist = fqdn_csv_list[i:] + out_list_normal[val] = ','.join(sublist[::step_size]) + +out_list_denormal = {} +for val, csvlist in out_list_normal.items(): + for i in csvlist.split(','): + if i != "": + out_list_denormal[i] = val + +if option == 'denormalize': + print json.dumps(out_list_denormal) +else: + print json.dumps(out_list_normal) Review comment: Addressed with commit ########## File path: infrastructure/ansible/sample.lab/Ansible.Dockerfile ########## @@ -0,0 +1,38 @@ +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +FROM centos:7.4.1708 +MAINTAINER Jonathan Gray +RUN yum -y install epel-release \ + && yum -y install \ + ansible \ + git \ + python-pip \ + python-devel \ + libxml2-devel \ + libxslt-devel \ + libffi-devel \ + openssl-devel \ + gcc \ + && yum clean all \ + && pip install --upgrade pip \ + && pip install --upgrade setuptools \ + && pip install --upgrade pyOpenSSL \ + && pip install --upgrade python-gilt \ + && pip install --upgrade paramiko \ + && pip install --upgrade Jinja2 Review comment: Addressed with commit ########## File path: docs/source/admin/index.rst ########## @@ -21,22 +21,29 @@ Traffic Control is distributed in source form for the developer, but also as a b When installing a complete CDN from scratch, a sample recommended order is: +#. Traffic Ops DB (Postgresql) +#. `InfluxDB [Optional] <https://github.com/influxdata/influxdb>`_ +#. Traffic Vault (Riak) [Optional] +#. Fake Origin [Optional] #. Traffic Ops -#. Traffic Vault (Riak) #. Traffic Portal +#. Initial Traffic Ops Dataset Setup [if applicable] #. Traffic Monitor #. Apache Traffic Server Mid-Tier Caches #. Apache Traffic Server Edge-Tier Caches +#. Grove [Optional] #. Traffic Router -#. Traffic Stats +#. `InfluxDB-relay [Optional] <https://github.com/influxdata/influxdb-relay>`_ +#. Traffic Stats [Optional] Once everything is installed, you will need to configure the servers to talk to each other. You will also need Origin server(s), from which the Mid-Tier Cache(s) will obtain content. An Origin server is simply an HTTP(S) server which serves the content you wish to cache on the CDN. .. toctree:: :maxdepth: 3 :glob: - traffic_ops.rst + environment_creation.rst + traffic_ops/* Review comment: Addressed with commit ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
