This is an automated email from the ASF dual-hosted git repository. ocket8888 pushed a commit to branch 5.0.x in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit 7f4b8ca79a05e605cfa631e9963e757e670225be Author: Zach Hoffman <[email protected]> AuthorDate: Tue Nov 17 13:00:04 2020 -0700 Remove unnecessary CDN in a Box waits (#5283) * Skip enroller initial data load when LOAD_TRAFFIC_OPS_DATA is not equal to true * Make DIG_IP_RETRY an inherited environment variable * Enroll Traffic Vault after Riak finishes starting up * Do not attempt to enroll Traffic Vault when running the Traffic Ops integration tests GitHub action * Add workflow-related files to Traffic Ops Go client/API integration tests paths filters * Add a period * Specify LOAD_TRAFFIC_OPS_DATA and DIG_IP_RETRY for trafficops-perl service only (cherry picked from commit 2b3e9cb71cf692e8531cb88f4354e9362f9a886b) --- .github/actions/to-integration-tests/entrypoint.sh | 2 +- .github/workflows/traffic ops.yml | 16 ++++++++++++---- docs/source/admin/quick_howto/ciab.rst | 4 ++++ docs/source/development/traffic_ops.rst | 3 +++ .../cdn-in-a-box/docker-compose.traffic-ops-test.yml | 7 ++++++- .../cdn-in-a-box/docker-compose.traffic-portal-test.yml | 4 ++++ infrastructure/cdn-in-a-box/docker-compose.yml | 3 +++ .../cdn-in-a-box/traffic_ops/set-to-ips-from-dns.sh | 1 - .../cdn-in-a-box/traffic_ops/trafficops-init.sh | 10 +++++++--- .../traffic_vault/poststart.d/02-add-search-schema.sh | 2 ++ infrastructure/cdn-in-a-box/traffic_vault/run.sh | 2 -- 11 files changed, 42 insertions(+), 12 deletions(-) diff --git a/.github/actions/to-integration-tests/entrypoint.sh b/.github/actions/to-integration-tests/entrypoint.sh index d3c4ff2..443e7f4 100755 --- a/.github/actions/to-integration-tests/entrypoint.sh +++ b/.github/actions/to-integration-tests/entrypoint.sh @@ -72,7 +72,7 @@ start_traffic_vault() { sed -i '0,/^update-ca-certificates/d' /etc/riak/prestart.d/00-config.sh; # Do not try to source to-access.sh - sed -i '/to-access\.sh/d' /etc/riak/{prestart.d,poststart.d}/* + sed -i '/to-access\.sh\|^to-enroll/d' /etc/riak/{prestart.d,poststart.d}/* BASH_LINES DOCKER_BUILDKIT=1 docker build "$ciab_dir" -f "${ciab_dir}/traffic_vault/Dockerfile" -t "$trafficvault" 2>&1 | diff --git a/.github/workflows/traffic ops.yml b/.github/workflows/traffic ops.yml index 80d674e..1a51795 100644 --- a/.github/workflows/traffic ops.yml +++ b/.github/workflows/traffic ops.yml @@ -20,17 +20,25 @@ name: Traffic Ops Go client/API integration tests on: push: paths: + - .github/actions/todb-init/** + - .github/actions/to-integration-tests/** + - '.github/workflows/traffic ops.yml' - GO_VERSION - - traffic_ops/traffic_ops_golang/**.go - - traffic_ops/testing/api/**.go + - infrastructure/cdn-in-a-box/traffic_vault/** - traffic_ops/*client/**.go + - traffic_ops/testing/api/**.go + - traffic_ops/traffic_ops_golang/**.go create: pull_request: paths: + - .github/actions/todb-init/** + - .github/actions/to-integration-tests/** + - '.github/workflows/traffic ops.yml' - GO_VERSION - - traffic_ops/traffic_ops_golang/**.go - - traffic_ops/testing/api/**.go + - infrastructure/cdn-in-a-box/traffic_vault/** - traffic_ops/*client/**.go + - traffic_ops/testing/api/**.go + - traffic_ops/traffic_ops_golang/**.go types: [opened, reopened, ready_for_review, synchronize] jobs: diff --git a/docs/source/admin/quick_howto/ciab.rst b/docs/source/admin/quick_howto/ciab.rst index 0f135ca..6a100d2 100644 --- a/docs/source/admin/quick_howto/ciab.rst +++ b/docs/source/admin/quick_howto/ciab.rst @@ -140,6 +140,8 @@ There also exist TP and TO integration tests containers. Both of these container sudo docker-compose -f docker-compose.traffic-ops-test.yml up +.. note:: If all CDN in a Box containers are started at once (example: ``docker-compose -f docker-compose.yml -f docker-compose.traffic-ops-test.yml up integration``), the :ref:`Enroller <ciab-enroller>` initial data load is skipped to prevent data conflicts with the :ref:`Traffic Ops API tests fixtures <dev-traffic-ops-fixtures>`. + variables.env """"""""""""" .. literalinclude:: ../../../../infrastructure/cdn-in-a-box/variables.env @@ -226,6 +228,8 @@ Advanced Usage ============== This section will be amended as functionality is added to the CDN in a Box project. +.. _ciab-enroller: + The Enroller ------------ The "enroller" began as an efficient way for Traffic Ops to be populated with data as CDN in a Box starts up. It connects to Traffic Ops as the "admin" user and processes files places in the docker volume shared between the containers. The enroller watches each directory within the ``/shared/enroller`` directory for new :file:`{filename}.json` files to be created there. These files must follow the format outlined in the API guide for the ``POST`` method for each data type, (e.g. for a ` [...] diff --git a/docs/source/development/traffic_ops.rst b/docs/source/development/traffic_ops.rst index bd3b7b6..38c9bb8 100644 --- a/docs/source/development/traffic_ops.rst +++ b/docs/source/development/traffic_ops.rst @@ -336,6 +336,9 @@ The integration tests are run using :manpage:`go-test(1)`, with two configuratio Specify the path to the `Test Configuration File`_. If not specified, it will attempt to read a file named ``traffic-ops-test.conf`` in the working directory. .. seealso:: `Configuring the Integration Tests`_ for a detailed explanation of the format of this configuration file. + +.. _dev-traffic-ops-fixtures: + .. option:: --fixtures FIXTURES Specify the path to a file containing static data for the tests to use. This should almost never be used, because many of the tests depend on the data having a certain content and structure. If not specified, it will attempt to read a file named ``tc-fixtures.json`` in the working directory. diff --git a/infrastructure/cdn-in-a-box/docker-compose.traffic-ops-test.yml b/infrastructure/cdn-in-a-box/docker-compose.traffic-ops-test.yml index 520df5f..2093e8a 100644 --- a/infrastructure/cdn-in-a-box/docker-compose.traffic-ops-test.yml +++ b/infrastructure/cdn-in-a-box/docker-compose.traffic-ops-test.yml @@ -38,8 +38,13 @@ services: volumes: - shared:/shared - ../../junit:/junit + + trafficops-perl: + environment: + DIG_IP_RETRY: ${DIG_IP_RETRY:-0} + LOAD_TRAFFIC_OPS_DATA: ${LOAD_TRAFFIC_OPS_DATA:-false} + volumes: junit: shared: external: false - diff --git a/infrastructure/cdn-in-a-box/docker-compose.traffic-portal-test.yml b/infrastructure/cdn-in-a-box/docker-compose.traffic-portal-test.yml index f285029..59ad76b 100644 --- a/infrastructure/cdn-in-a-box/docker-compose.traffic-portal-test.yml +++ b/infrastructure/cdn-in-a-box/docker-compose.traffic-portal-test.yml @@ -41,6 +41,10 @@ services: - shared:/shared - ../../junit:/junit + trafficops-perl: + environment: + DIG_IP_RETRY: ${DIG_IP_RETRY:-0} + volumes: shared: external: false diff --git a/infrastructure/cdn-in-a-box/docker-compose.yml b/infrastructure/cdn-in-a-box/docker-compose.yml index a78c52b..943ee4a 100644 --- a/infrastructure/cdn-in-a-box/docker-compose.yml +++ b/infrastructure/cdn-in-a-box/docker-compose.yml @@ -89,6 +89,9 @@ services: domainname: infra.ciab.test env_file: - variables.env + environment: + DIG_IP_RETRY: ${DIG_IP_RETRY:-10} + LOAD_TRAFFIC_OPS_DATA: ${LOAD_TRAFFIC_OPS_DATA:-true} hostname: trafficops-perl image: trafficops-perl # TODO: change to expose: "60443" to limit to containers diff --git a/infrastructure/cdn-in-a-box/traffic_ops/set-to-ips-from-dns.sh b/infrastructure/cdn-in-a-box/traffic_ops/set-to-ips-from-dns.sh index 1df6a8f..dc73ee9 100755 --- a/infrastructure/cdn-in-a-box/traffic_ops/set-to-ips-from-dns.sh +++ b/infrastructure/cdn-in-a-box/traffic_ops/set-to-ips-from-dns.sh @@ -40,7 +40,6 @@ service_ips="${gateway_ip}" service_ip6s="${gateway_ip6}" INTERFACE=$(ip link | awk '/\<UP\>/ && !/LOOPBACK/ {sub(/@.*/, "", $2); print $2}') NETMASK=$(route | awk -v INTERFACE=$INTERFACE '$8 ~ INTERFACE && $1 !~ "default" {print $3}') -DIG_IP_RETRY=10 for service_name in $service_names; do service_fqdn="${service_name}.${service_domain}" diff --git a/infrastructure/cdn-in-a-box/traffic_ops/trafficops-init.sh b/infrastructure/cdn-in-a-box/traffic_ops/trafficops-init.sh index b0dc2b4..11deb9b 100755 --- a/infrastructure/cdn-in-a-box/traffic_ops/trafficops-init.sh +++ b/infrastructure/cdn-in-a-box/traffic_ops/trafficops-init.sh @@ -141,7 +141,11 @@ traffic_router_zonemanager_timeout() { mv "$modified_crconfig" $crconfig_path; } -traffic_router_zonemanager_timeout +if [[ "$LOAD_TRAFFIC_OPS_DATA" == true ]]; then + traffic_router_zonemanager_timeout -# Load required data at the top level -load_data_from /traffic_ops_data + # Load required data at the top level + load_data_from /traffic_ops_data +else + touch "$ENROLLER_DIR/initial-load-done" +fi diff --git a/infrastructure/cdn-in-a-box/traffic_vault/poststart.d/02-add-search-schema.sh b/infrastructure/cdn-in-a-box/traffic_vault/poststart.d/02-add-search-schema.sh index 512f479..dcf2037 100644 --- a/infrastructure/cdn-in-a-box/traffic_vault/poststart.d/02-add-search-schema.sh +++ b/infrastructure/cdn-in-a-box/traffic_vault/poststart.d/02-add-search-schema.sh @@ -21,3 +21,5 @@ curl -kvs -XPUT -H 'Content-Type:application/xml' "https://$TV_ADMIN_USER:$TV_AD curl -kvs -XPUT -H 'Content-Type:application/json' "https://$TV_ADMIN_USER:$TV_ADMIN_PASSWORD@$TV_FQDN:$TV_HTTPS_PORT/search/index/sslkeys"; -d '{"schema":"sslkeys"}' curl -kvs -XPUT -H 'Content-Type:application/json' "https://$TV_ADMIN_USER:$TV_ADMIN_PASSWORD@$TV_FQDN:$TV_HTTPS_PORT/buckets/ssl/props"; -d'{"props":{"search_index":"sslkeys"}}' + +to-enroll "tv" ALL "" "8088" "8088" || (while true; do echo "enroll failed."; sleep 3 ; done) diff --git a/infrastructure/cdn-in-a-box/traffic_vault/run.sh b/infrastructure/cdn-in-a-box/traffic_vault/run.sh index a7667e0..5585d0c 100755 --- a/infrastructure/cdn-in-a-box/traffic_vault/run.sh +++ b/infrastructure/cdn-in-a-box/traffic_vault/run.sh @@ -26,6 +26,4 @@ TO_URL=https://${TO_FQDN}:${TO_PORT} TO_USER=$TV_USER TO_PASSWORD=$TV_PASSWORD -to-enroll "tv" ALL "" "8088" "8088" || (while true; do echo "enroll failed."; sleep 3 ; done) - ${RIAK_HOME}/riak-cluster.sh
