dangogh closed pull request #2965: Config compare test
URL: https://github.com/apache/trafficcontrol/pull/2965
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/docs/source/tools/compare.rst b/docs/source/tools/compare.rst
index 003e4167f..165d7c692 100644
--- a/docs/source/tools/compare.rst
+++ b/docs/source/tools/compare.rst
@@ -82,6 +82,8 @@ usage: genConfigRoutes.py [-h] [-k] [-v] InstanceA InstanceB
LoginA [LoginB]
.. note:: If you're using a CDN-in-a-Box environment for testing, it's likely
that you'll need the ``-k``/``--insecure`` option if you're outside the Docker
network
+.. _compare-genConfigRoutes-positional_parameters:
+
.. table:: Positional Parameters (In Order)
+-----------+---------------------------------------------------------------------------------------------------------------------------------------+
@@ -97,6 +99,8 @@ usage: genConfigRoutes.py [-h] [-k] [-v] InstanceA InstanceB
LoginA [LoginB]
| | If not given, LoginA will be re-used for the second
connection (default: None)
|
+-----------+---------------------------------------------------------------------------------------------------------------------------------------+
+.. note:: The full behaviour of the ``LoginB`` parameter described in
:ref:`compare-genConfigRoutes-positional_parameters` is such that supports not
only a fully missing authentication credentials pair, but also a blank string
for each of the pair members. This means that you can, for instance, give it
the testing pair 'admin:' to use the testing username 'admin', but default to
the password used by the reference user. Or, another example is that passing
``LoginB`` as simple ``:`` will result in the same behaviour as not passing it
all.
+
The genConfigRoutes.py script will output list of unique API routes (relative
to the desired Traffic Ops URL) that point to generated configuration files for
a sample set of servers common to both Traffic Ops instances. The results are
printed to stdout, making the output perfect for piping directly into
``compare`` like so:
.. code-block:: shell
@@ -104,3 +108,29 @@ The genConfigRoutes.py script will output list of unique
API routes (relative to
./genConfigRoutes.py https://trafficopsA.example.test
https://trafficopsB.example.test username:password | ./compare
\... assuming the proper environment variables have been set for ``compare``.
+
+Usage with Docker
+=================
+A Dockerfile is provided to run tests on a pair of instances given the
configuration environment variables necessary. This will generate configuration
file routes using ``genConfigRoutes.py``, and add them to whatever is already
contained in ``traffic_ops/testing/compare/testroutes.txt``, then run the
``compare`` tool on the final API route list. Build artifacts (i.e. anything
output files created by the `compare` tool) are placed in the `/artifacts/`
directory on the container. To retrieve these results, the use of a volume is
recommended. The build context *must* be at the root of the Traffic Control
repository, as the tools have dependencies on the Traffic Control clients. In
order to use the container, the following environment variables must be defined
for the container at runtime:
+
+TO_URL
+ The URL of the reference Traffic Ops instance
+TO_USER
+ The username to authenticate with the reference Traffic Ops instance
+TO_PASSWORD
+ The password to authenticate with the reference Traffic Ops instance
+TEST_URL
+ The URL of the testing Traffic Ops instance
+TEST_USER
+ The username to authenticate with the testing Traffic Ops instance
+TEST_PASSWORD
+ The password to authenticate with the testing Traffic Ops instance
+
+.. code-block:: shell
+ :caption: Sample Script to Build and Run
+ sudo docker build . -f traffic_ops/testing/compare/Dockerfile -t
compare:latest
+ sudo docker run -v $PWD/artifacts:/artifacts -e TO_URL="$TO_URL" -e
TEST_URL="$TEST_URL" -e TO_USER="admin" -e TO_PASSWORD="twelve" -e
TEST_USER="admin" -e TEST_PASSWORD="twelve" compare:latest
+
+.. note:: The above code example assumes that the environment variables
``TO_URL`` and ``TEST_URL`` refer to the URL of the reference Traffic Ops
instance and the URL of the test Traffic Ops instance, respectively (including
port numbers). It also uses credentials suitable for logging into a stock
:ref:`ciab` instance.
+
+.. note:: Unlike using the ``genRoutesConfig.py`` script and/or the
``compare`` on their own, *all* of the variables must be defined, even if they
are duplicates.
diff --git a/traffic_control/clients/python/common/restapi.py
b/traffic_control/clients/python/common/restapi.py
index 177e66e57..4fe55184b 100644
--- a/traffic_control/clients/python/common/restapi.py
+++ b/traffic_control/clients/python/common/restapi.py
@@ -36,7 +36,11 @@
# Python 2 to Python 3 Compatibility
import requests.compat as compat
from builtins import str
-from future.utils import iteritems
+
+try:
+ from future.utils import iteritems
+except ImportError:
+ iteritems = lambda x: x.items()
logger = logging.getLogger(__name__)
diff --git a/traffic_ops/testing/compare/Dockerfile
b/traffic_ops/testing/compare/Dockerfile
new file mode 100644
index 000000000..e11041734
--- /dev/null
+++ b/traffic_ops/testing/compare/Dockerfile
@@ -0,0 +1,48 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+############################################################
+# Dockerfile to build Docker images for testing via the 'compare' tool
+# Based on Alpine Linux
+############################################################
+
+FROM golang:alpine
+
+ENV GOPATH /go/
+RUN mkdir -p
/go/src/github.com/apache/trafficcontrol/traffic_control/clients/python\
+ /go/src/github.com/apache/trafficcontrol/lib\
+ /go/src/github.com/apache/trafficcontrol/traffic_ops/testing/compare\
+
/go/src/github.com/apache/trafficcontrol/traffic_ops/vendor/github.com/kelseyhightower\
+ /artifacts
+
+RUN apk update
+RUN apk add python3 git
+RUN python3 -m ensurepip && python3 -m pip install --upgrade pip && python3 -m
pip install requests munch
+
+ADD traffic_control/clients/python
/go/src/github.com/apache/trafficcontrol/traffic_control/clients/python/
+ADD lib /go/src/github.com/apache/trafficcontrol/lib
+ADD vendor /go/src/github.com/apache/trafficcontrol/vendor
+ADD traffic_ops/vendor/github.com/kelseyhightower
/go/src/github.com/apache/trafficcontrol/traffic_ops/vendor/github.com/kelseyhightower
+ADD traffic_ops/testing/compare
/go/src/github.com/apache/trafficcontrol/traffic_ops/testing/compare
+
+WORKDIR /go/src/github.com/apache/trafficcontrol/traffic_ops/testing/compare/
+RUN go get -v ./...
+RUN go build .
+RUN cp testroutes.txt /artifacts/
+
+CMD ./genConfigRoutes.py -k $TO_URL $TEST_URL $TO_USER:$TO_PASSWORD
$TEST_USER:$TEST_PASSWORD -l INFO 2>&1 >>/artifacts/testroutes.txt | tee
/artifacts/genRoutesConfig.log &&\
+ ./compare --ref_url=$TO_URL --test_url=$TEST_URL --ref_user=$TO_USER
--ref_passwd=$TO_PASSWORD --test_user=$TEST_USER --test_passwd=$TEST_PASSWORD
-r /artifacts </artifacts/testroutes.txt
diff --git a/traffic_ops/testing/compare/README.md
b/traffic_ops/testing/compare/README.md
index e765384a9..f7fc302fa 100644
--- a/traffic_ops/testing/compare/README.md
+++ b/traffic_ops/testing/compare/README.md
@@ -1,20 +1,20 @@
<!--
- Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
-->
The Compare Tool
@@ -92,6 +92,18 @@ The genConfigRoutes.py script will output list of unique API
routes (relative to
[1] Theoretically, if you downloaded the Traffic Control repository properly
(into `$GOPATH/src/github.com/apache/trafficcontrol`), this will already be
satisfied.
+Usage With Docker
+-----------------
+A Dockerfile is provided to run tests on a pair of instances given the
configuration environment variables necessary. This will generate configuration
file routes using `genConfigRoutes.py`, and add them to whatever is already
contained in [testroutes.txt](./testroutes.txt), then run the `compare` tool on
the final API route list. Build artifacts (i.e. anything output files created
by the `compare` tool) are placed in the `/artifacts/` directory on the
container. To retrieve these results, the use of a volume is recommended. The
build context _must_ be at the root of the Traffic Control repository, as the
tools have dependencies on the Traffic Control clients.
+
+```bash
+sudo docker build . -f traffic_ops/testing/compare/Dockerfile -t compare:latest
+sudo docker run -v $PWD/artifacts:/artifacts -e TO_URL="$TO_URL" -e
TEST_URL="$TEST_URL" -e TO_USER="admin" -e TO_PASSWORD="twelve" -e
TEST_USER="admin" -e TEST_PASSWORD="twelve" compare:latest
+```
+
+!!! note
+ The above code example assumes that the environment variables `TO_URL`
and `TEST_URL` refer to the URL of the reference Traffic Ops instance and the
URL of the test Traffic Ops instance, respectively (including port numbers). It
also uses credentials suitable for logging into a stock CDN-in-a-Box instance.
+
Further Information
-------------------
diff --git a/traffic_ops/testing/compare/genConfigRoutes.py
b/traffic_ops/testing/compare/genConfigRoutes.py
index c4720e691..0725c8892 100755
--- a/traffic_ops/testing/compare/genConfigRoutes.py
+++ b/traffic_ops/testing/compare/genConfigRoutes.py
@@ -170,8 +170,25 @@ def main(kwargs:argparse.Namespace) -> int:
instanceA = kwargs.InstanceA
instanceB = kwargs.InstanceB
- loginA = kwargs.LoginA.split(':')
- loginB = kwargs.LoginB.split(':') if kwargs.LoginB else loginA
+
+ try:
+ loginA = kwargs.LoginA.split(':')
+ loginA = (loginA[0], ':'.join(loginA[1:]))
+ except (KeyError, IndexError) as e:
+ logging.critical("Bad username/password pair: '%s' (hint: try
-h/--help)", kwargs.LoginA)
+ return 1
+
+ loginB = loginA
+
+ try:
+ if kwargs.LoginB:
+ loginB = kwargs.LoginB.split(':')
+ loginB = (loginB[0], ':'.join(loginB[1:]))
+ loginB = (loginB[0] if loginB[0] else loginA[0],
loginB[1] if loginB[1] else loginA[1])
+ except (KeyError, IndexError) as e:
+ logging.critical("Bad username/password pair: '%s' (hint: try
-h/--help)", kwargs.LoginB)
+ return 1
+
verify = not kwargs.insecure
# Peel off all schemas
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
