zrhoffman opened a new pull request #4758: URL: https://github.com/apache/trafficcontrol/pull/4758
<!-- ************ STOP!! ************ If this Pull Request is intended to fix a security vulnerability, DO NOT submit it! Instead, contact the Apache Software Foundation Security Team at [email protected] and follow the guidelines at https://www.apache.org/security/ regarding vulnerability disclosure. --> ## What does this PR (Pull Request) do? <!-- Explain the changes you made here. If this fixes an Issue, identify it by replacing the text in the checkbox item with the Issue number e.g. - [x] This PR fixes #9001 OR is not related to any Issue ^ This will automatically close Issue number 9001 when the Pull Request is merged (The '#' is important). Be sure you check the box properly, see the "The following criteria are ALL met by this PR" section for details. --> - - [x] This PR is not related to any Issue <!-- You can check for an issue here: https://github.com/apache/trafficcontrol/issues --> - This PR lets you build the ATC RPMs, target for CentOS 7, from any OS without needing Docker. See the table below for time comparisons - This PR includes various compatibility improvements to the build scripts: - Makes the build scripts run in Bourne shell (`sh`) instead of Bash for POSIX compatibility - Uses `realpath` instead of `readlink` where possible - Eliminates Perl as a build-time dependency for the Traffic Ops RPM - Fixes a bug [from 4cf9529](https://github.com/apache/trafficcontrol/commit/4cf95296ed#diff-03abf91e897fb7b93c1f1da538361657L106) where failing the environment check did not stop the build script - Statically links the Go binaries, which ends up making them a little smaller (see table) - Expands Traffic Portal and Traffic Router *Software Requirements* sections in the docs ### Comparison of time taken to build each component (RPM or tar.gz, depending on the component) |docs|grove|grovetccfg|source|traffic_monitor|traffic_ops|traffic_portal|traffic_router|traffic_stats| |----|----:|---------:|-----:|--------------:|----------:|-------------:|-------------:|------------:| |Docker on macOS|`02:54.01`|`00:53.44`|`00:44.94`|`00:43.69`|`00:50.65`|`01:11.01`|`01:49.90`|`01:44.56`|`00:55.07`| |Docker on Linux|`02:11.94`|`00:15.17`|`00:13.53`|`00:06.10`|`00:15.39`|`00:24.18`|`01:01.34`|`00:49.34`|`00:16.31`| |Natively on macOS|`01:53.61`|`00:03.38`|`00:03.97`|`00:04.36`|`00:04.93`|`00:10.40`|`00:45.23`|`00:33.98`|`00:07.98`| |Natively on Linux|`01:14.69`|`00:02.68`|`00:03.18`|`00:02.91`|`00:03.37`|`00:07.65`|`00:27.99`|`00:18.32`|`00:06.12`| Some comments about the time comparisons table: * *Docker* refers to `./pkg -v [component]`, *Natively* refers to `build/clean_build [component]` * For components built natively: The repo is already copied to `/tmp/go/src/github.com/apache/trafficcontrol` , so the `rsync` command in `clean_build.sh` to copy the repo there only affects files that have changed since the last native build. * Each `./pkg` command bootstraps `docker-compose`, which takes ~1.6 seconds. * For components built using Docker: Because `/tmp/go/src/github.com/apache/trafficcontrol` is not a volume, the `rsync` command in `clean_build.sh` always copies the entire repo. - Running Docker on Linux, this `rsync` takes on average ~3.2 seconds. - In Docker for Mac, it takes ~34 seconds. ### Binary size: Static linking vs dynamic linking Since this PR makes the Go binaries statically linked, whereas before they were dynamically linked, I expected the binaries to be a little larger, but it ends up saving a very small amount of space instead. |grove|grovetccfg|traffic_monitor|traffic_ops_golang|atstccfg|traffic_stats| |----:|---------:|--------------:|-----------------:|-------:|------------:| |Dynamic linking (KiB)|`7261`|`9358`|`11189`|`18473`|`10077`|`9983`| |Static linking (KiB)|`7223`|`9300`|`11127`|`18404`|`10019`|`9917`| |**Space saved (KiB)**|`38`|`58`|`62`|`69`|`58`|`66`| Some comments about the static vs. dynamic linking table: * Dynamically linked binaries were built with `go build -ldflags '-s -w'` and were then stripped of symbols (`strip [binary name]`) * Statically linked binaries were built with `go build -ldflags '-s -w' -tags 'osusergo netgo'` and were then stripped of symbols (`strip [binary name]`) Besides being a tiny bit smaller, the static binaries will load trivially faster at runtime. However, our statically-linked Go binaries on average take about 1.2 seconds longer to build each. ## Which Traffic Control components are affected by this PR? <!-- Please delete all components from this list that are NOT affected by this Pull Request. Also, feel free to add the name of a tool or script that is affected but not on the list. Additionally, if this Pull Request does NOT affect documentation, please explain why documentation is not required. --> - CDN in a Box - Documentation - Grove - Traffic Monitor - Traffic Ops - Traffic Ops ORT - Traffic Portal - Traffic Router - Traffic Stats ## What is the best way to verify this PR? <!-- Please include here ALL the steps necessary to test your Pull Request. If it includes tests (and most should), outline here the steps needed to run the tests. If not, lay out the manual testing procedure and please explain why tests are unnecessary for this Pull Request. --> * Verify that the target OS of the RPMs is Linux ```shell rpm -qp --qf '%{os}\n' *.rpm ``` * Verify that the RPM compression format is XZ ```shell rpm -qp --qf '%{payloadcompressor}\n' *.rpm ``` * Verify that all of the packaged Go binaries are statically linked (have no *dynamic* section). Example for `traffic_ops_golang`: ```shell docker run --rm -v$(pwd):/mnt centos:7 readelf -d /mnt/traffic_ops_golang ``` Expected output is ``` There is no dynamic section in this file. ``` * In the CDN-in-a-Box directory, run `make` to build the RPMs and verify with CiaB that they work * - On `docs/source/development/building.rst` under *Build the RPMs Natively*, follow the *Install the Dependencies* instructions for your OS - In the CiB directory, run `make native` to build the RPMs and verify with CiaB that they work * - For `./pkg -v` (Docker), verify the RPMs and gzipped TARs that CDN-in-a-Box does not use: * apache-trafficcontrol-4.2.0-docs.tar.gz * apache-trafficcontrol-4.2.0.tar.gz * grove-0.2-#####.XXXXXXXX.x86_64.rpm * grovetccfg-0.2-#####.XXXXXXXX.x86_64.rpm * traffic_ops-4.2.0-#####.XXXXXXXX.el7.src.rpm * traffic_ops_ort-4.2.0-#####.XXXXXXXX.el7.src.rpm * traffic_ops_ort-4.2.0-#####.XXXXXXXX.el7.x86_64.rpm * traffic_portal-4.2.0-#####.XXXXXXXX.el7.src.rpm * traffic_stats-4.2.0-#####.XXXXXXXX.el7.src.rpm - For `build/clean_build.sh` (native), verify the RPMs and gzipped TARs that CDN-in-a-Box does not use ## If this is a bug fix, what versions of Traffic Control are affected? <!-- If this PR fixes a bug, please list here all of the affected versions - to the best of your knowledge. It's also pretty helpful to include a commit hash of where 'master' is at the time this PR is opened (if it affects master), because what 'master' means will change over time. For example, if this PR fixes a bug that's present in master (at commit hash '1df853c8'), in v4.0.0, and in the current 4.0.1 Release candidate (e.g. RC1), then this list would look like: - master (1df853c8) - 4.0.0 - 4.0.1 (RC1) If you don't know what other versions might have this bug, AND don't know how to find the commit hash of 'master', then feel free to leave this section blank (or, preferably, delete it entirely). --> For the environment check bugfix: - master (8d2e53ec4d) - 4.0.0 - 4.1.0 (RC1) Otherwise, not a bug fix. ## The following criteria are ALL met by this PR <!-- Check the boxes to signify that the associated statement is true. To "check a box", replace the space inside of the square brackets with an 'x'. e.g. - [ x] <- Wrong - [x ] <- Wrong - [] <- Wrong - [*] <- Wrong - [x] <- Correct! --> - [x] This PR does not contain tests because our CI pipeline tests the build system - [x] This PR includes documentation - [x] This PR includes an update to CHANGELOG.md - [x] This PR includes any and all required license headers - [x] This PR does not include a database migration - [x] This PR **DOES NOT FIX A SERIOUS SECURITY VULNERABILITY** (see [the Apache Software Foundation's security guidelines](https://www.apache.org/security/) for details) <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
