zrhoffman commented on a change in pull request #6532: URL: https://github.com/apache/trafficcontrol/pull/6532#discussion_r799064897
########## File path: dev/traffic_ops/Dockerfile ########## @@ -0,0 +1,38 @@ +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +FROM alpine:latest AS certbuilder +RUN apk add --no-cache openssl +RUN openssl genrsa -passout pass:x -out server.pass.key 2048 && \ + openssl rsa -passin pass:x -in server.pass.key -out server.key && \ + rm server.pass.key && \ + openssl req -new -key server.key -out server.csr \ + -subj "/C=US/ST=CO/L=Denver/O=Apache/OU=Traffic Control/CN=trafficops.dev.ciab.test" && \ + openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt && \ + openssl rand 32 | base64 > /aes.key + +FROM alpine:latest AS trafficops-dev + +ENV TC="/root/go/src/github.com/apache/trafficcontrol/" +VOLUME /root/go/src/github.com/apache/trafficcontrol +ENV ADMIN="$TC/traffic_ops/app/db/admin" +EXPOSE 443 6444 + +COPY --from=certbuilder /server.key /server.crt /aes.key / +RUN apk add --no-cache make inotify-tools go postgresql-client && go install github.com/go-delve/delve/cmd/dlv@latest && ln -s /root/go/bin/dlv /usr/bin/dlv Review comment: Alpine currently downloads Go version 1.17.4. In order for debugging to work properly, the TO image should use the Go version specified in the `GO_VERSION` file. https://github.com/apache/trafficcontrol/blob/abfbb3483afbb3deedb488e872d21d8c0ba6c94a/GO_VERSION#L1 ########## File path: dev/traffic_monitor/Dockerfile ########## @@ -0,0 +1,23 @@ +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +FROM alpine:latest AS trafficmonitor-dev + +ENV TC=/root/go/src/github.com/apache/trafficcontrol +VOLUME /root/go/src/github.com/apache/trafficcontrol +EXPOSE 80 81 + +RUN apk add --no-cache inotify-tools go && go install github.com/go-delve/delve/cmd/dlv@latest && ln -s /root/go/bin/dlv /usr/bin/dlv Review comment: Alpine currently downloads Go version 1.17.4. In order for debugging to work properly, the TM image should use the Go version specified in the `GO_VERSION` file. https://github.com/apache/trafficcontrol/blob/abfbb3483afbb3deedb488e872d21d8c0ba6c94a/GO_VERSION#L1 ########## File path: dev/traffic_router/run.sh ########## @@ -0,0 +1,41 @@ +#!/bin/sh +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +set -o errexit + +cd "$TC/traffic_router" + +mvn -Dmaven.test.skip=true compile -P \!rpm-build +mvn -Dmaven.test.skip=true package -P \!rpm-build + +cp core/target/ROOT.war /opt/tomcat/webapps/ +ln -s $TC/dev/traffic_router/conf/* /opt/tomcat/conf/ Review comment: Modifying `/opt/tomcat` beyond removing demo stuff is not ideal. Once the `CATALINA_BASE` environment variable is set to `/opt/traffic_router` and `/opt/traffic_router` is symlinked to `${TC}/dev/traffic_router`, Tomcat can load `server.xml` (and the other config files) from the `${TC}/dev/traffic_router/conf` directory. ########## File path: dev/traffic_router/run.sh ########## @@ -0,0 +1,41 @@ +#!/bin/sh +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +set -o errexit + +cd "$TC/traffic_router" + +mvn -Dmaven.test.skip=true compile -P \!rpm-build +mvn -Dmaven.test.skip=true package -P \!rpm-build + +cp core/target/ROOT.war /opt/tomcat/webapps/ +ln -s $TC/dev/traffic_router/conf/* /opt/tomcat/conf/ + +/opt/tomcat/bin/catalina.sh jpda run Review comment: > <pre>Using CATALINA_BASE: /opt/tomcat > Using CATALINA_HOME: /opt/tomcat > Using CATALINA_TMPDIR: /opt/tomcat/temp > Using JRE_HOME: /usr/lib/jvm/java-11-openjdk > Using CLASSPATH: /opt/tomcat/bin/bootstrap.jar:/opt/tomcat/bin/tomcat-juli.jar > Using CATALINA_OPTS:</pre></quote> The way Traffic Router is set up, `CATALINA_BASE` is supposed to be `/opt/traffic_router`: https://github.com/apache/trafficcontrol/blob/abfbb3483afbb3deedb488e872d21d8c0ba6c94a/traffic_router/core/src/main/lib/systemd/system/traffic_router.service#L24-L25 `CATALINA_BASE` should be equal to the `deploy.dir` system property. It is defined here: https://github.com/apache/trafficcontrol/blob/abfbb3483afbb3deedb488e872d21d8c0ba6c94a/traffic_router/pom.xml#L34 Among other things, `deploy.dir` is used to find the property files. https://github.com/apache/trafficcontrol/blob/abfbb3483afbb3deedb488e872d21d8c0ba6c94a/traffic_router/core/src/main/resources/applicationProperties.xml#L28-L32 If you can find a way to change `deploy.dir` to `${TC}/dev/traffic_router`, that would be idea. Otherwise, `CATALINA_BASE` should be set to `/opt/traffic_router`, with `/opt/traffic_router` being a symlink to `${TC}/dev/traffic_router`. ########## File path: dev/traffic_router/run.sh ########## @@ -0,0 +1,41 @@ +#!/bin/sh +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +set -o errexit + +cd "$TC/traffic_router" + +mvn -Dmaven.test.skip=true compile -P \!rpm-build +mvn -Dmaven.test.skip=true package -P \!rpm-build + +cp core/target/ROOT.war /opt/tomcat/webapps/ Review comment: 1. We don't need a `.war` file 2. Once the `CATALINA_BASE` environment variable is set to `/opt/traffic_router` and `/opt/traffic_router` is symlinked to `${TC}/dev/traffic_router`, Tomcat will read webapps from `${TC}/dev/traffic_router/webapps`. If it finds in there a relative symlink to `traffic_router/core/target/ROOT`, it can load TR without bothering with a `.war`. ########## File path: dev/traffic_router/Dockerfile ########## @@ -0,0 +1,31 @@ +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +FROM alpine:latest AS trafficrouter-dev + +ENV TC=/root/go/src/github.com/apache/trafficcontrol +VOLUME /root/go/src/github.com/apache/trafficcontrol +ENV JPDA_OPTS="-agentlib:jdwp=transport=dt_socket,address=5005,server=y,suspend=n" JAVA_HOME=/usr/lib/jvm/java-11-openjdk M2_HOME=/root/.m2/ Review comment: M2_HOME should be a bind mount to the gitignored `.m2` directory used by the TR builder image https://github.com/apache/trafficcontrol/blob/abfbb3483afbb3deedb488e872d21d8c0ba6c94a/infrastructure/docker/build/docker-compose.yml#L111 ########## File path: dev/t3c/Dockerfile ########## @@ -0,0 +1,49 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +FROM alpine:latest AS traffic-server-builder + +RUN apk add --no-cache build-base perl libexecinfo-dev pcre-dev libressl-dev libtool linux-headers openssl-dev zlib-dev +ADD https://downloads.apache.org/trafficserver/trafficserver-9.1.1.tar.bz2 / +RUN tar -xf trafficserver-9.1.1.tar.bz2 && cd trafficserver-9.1.1 && mkdir /ats && ./configure --prefix / --enable-experimental-plugins && make -j && make install && ls -R /ats + +FROM alpine:latest AS t3c-dev + +ENV TC="/root/go/src/github.com/apache/trafficcontrol/" +VOLUME /root/go/src/github.com/apache/trafficcontrol +EXPOSE 80 8081 + +COPY --from=traffic-server-builder /bin/traffic_cache_tool /bin/traffic_crashlog /bin/traffic_ctl /bin/traffic_layout /bin/traffic_logcat /bin/traffic_logstats /bin/traffic_manager /bin/traffic_server /bin/traffic_via /bin/trafficserver /bin/tspush /bin/tsxs /bin/ +COPY --from=traffic-server-builder /etc/trafficserver /etc/trafficserver +COPY --from=traffic-server-builder /include/ts /include/ts +COPY --from=traffic-server-builder /include/tscpp /include/tscpp +COPY --from=traffic-server-builder /lib/perl5 /lib/perl5 +COPY --from=traffic-server-builder /lib/pkgconfig/trafficserver.pc /lib/pkgconfig/trafficserver.pc +COPY --from=traffic-server-builder /lib/libtscore.la /lib/libtscore.so /lib/libtscore.so.9 /lib/libtscore.so.9.1.1 /lib/libtscppapi.la /lib/libtscppapi.so /lib/libtscppapi.so.9 /lib/libtscppapi.so.9.1.1 /lib/libtscpputil.la /lib/libtscpputil.so /lib/libtscpputil.so.9 /lib/libtscpputil.so.9.1.1 /lib/libtsmgmt.la /lib/libtsmgmt.so /lib/libtsmgmt.so.9 /lib/libtsmgmt.so.9.1.1 /lib/plugin_init_fail.la /lib/plugin_init_fail.so /lib/plugin_instinit_fail.la /lib/plugin_instinit_fail.so /lib/plugin_missing_deleteinstance.la /lib/plugin_missing_deleteinstance.so /lib/plugin_missing_doremap.la /lib/plugin_missing_doremap.so /lib/plugin_missing_init.la /lib/plugin_missing_init.so /lib/plugin_missing_newinstance.la /lib/plugin_missing_newinstance.so /lib/plugin_required_cb.la /lib/plugin_required_cb.so /lib/plugin_testing_calls.la /lib/plugin_testing_calls.so /lib/plugin_v1.la /lib/plugin_v1.so /lib/plugin_v2.la /lib/plugin_v2.so /lib/ +COPY --from=traffic-server-builder /libexec/trafficserver /libexec/trafficserver +COPY --from=traffic-server-builder /share/man /share/man +RUN mkdir /share/trafficserver && mkdir -p /var/log/trafficserver && mkdir /var/trafficserver + +RUN apk add --no-cache make inotify-tools go pcre libexecinfo && \ Review comment: Alpine currently downloads Go version 1.17.4. In order for debugging to work properly, the t3c image should use the Go version specified in the `GO_VERSION` file. https://github.com/apache/trafficcontrol/blob/abfbb3483afbb3deedb488e872d21d8c0ba6c94a/GO_VERSION#L1 ########## File path: dev/traffic_router/Dockerfile ########## @@ -0,0 +1,31 @@ +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +FROM alpine:latest AS trafficrouter-dev + +ENV TC=/root/go/src/github.com/apache/trafficcontrol +VOLUME /root/go/src/github.com/apache/trafficcontrol +ENV JPDA_OPTS="-agentlib:jdwp=transport=dt_socket,address=5005,server=y,suspend=n" JAVA_HOME=/usr/lib/jvm/java-11-openjdk M2_HOME=/root/.m2/ +EXPOSE 3053:53/tcp 3053:53/udp 3080:80 3443:443 3333:3333 2222:3443 5005:5005 + +RUN apk add --no-cache openjdk11 inotify-tools maven && ln -s /usr/lib/jvm/java-11-openjdk/bin/jdb /bin/jdb + +ADD https://archive.apache.org/dist/tomcat/tomcat-9/v9.0.43/bin/apache-tomcat-9.0.43.tar.gz /opt/tomcat.tgz +RUN cd /opt && \ + tar -xf tomcat.tgz && \ + mv apache-tomcat-* tomcat && \ + rm -rf tomcat.tgz tomcat/webapps/* tomcat/conf/web.xml tomcat/conf/server.xml tomcat/conf/logging.properties /usr/share/java/maven-3/conf/settings.xml Review comment: The `-rf` should be just `-r` so as not to mask attempts to delete nonexistent files and directories ########## File path: dev/traffic_router/run.sh ########## @@ -0,0 +1,41 @@ +#!/bin/sh +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +set -o errexit + +cd "$TC/traffic_router" + +mvn -Dmaven.test.skip=true compile -P \!rpm-build +mvn -Dmaven.test.skip=true package -P \!rpm-build + +cp core/target/ROOT.war /opt/tomcat/webapps/ +ln -s $TC/dev/traffic_router/conf/* /opt/tomcat/conf/ + +/opt/tomcat/bin/catalina.sh jpda run Review comment: TR exits immediately after starting. The contents of `/opt/tomcat/logs/catalina.2022-02-04.log`: ```java 04-Feb-2022 00:41:39.532 SEVERE [main] org.apache.tomcat.util.digester.Digester.startElement Begin event threw exception java.lang.ClassNotFoundException: org.apache.traffic_control.traffic_router.tomcat.TomcatLifecycleListener at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) at org.apache.tomcat.util.digester.ObjectCreateRule.begin(ObjectCreateRule.java:104) at org.apache.catalina.startup.ListenerCreateRule.begin(ListenerCreateRule.java:70) at org.apache.tomcat.util.digester.Digester.startElement(Digester.java:1293) at java.xml/com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:510) at java.xml/com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:183) at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanStartElement(XMLDocumentFragmentScannerImpl.java:1377) at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2710) at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:605) at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:534) at java.xml/com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:888) at java.xml/com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:824) at java.xml/com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141) at java.xml/com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1216) at java.xml/com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:635) at org.apache.tomcat.util.digester.Digester.parse(Digester.java:1551) at org.apache.catalina.startup.Catalina.parseServerXml(Catalina.java:617) at org.apache.catalina.startup.Catalina.load(Catalina.java:709) at org.apache.catalina.startup.Catalina.load(Catalina.java:746) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:302) at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:472) ``` So Tomcat cannot find `org.apache.traffic_control.traffic_router.tomcat.TomcatLifecycleListener`, which `server.xml` tells it to load: https://github.com/apache/trafficcontrol/blob/7b30bab1a6931927f092f78c74b5ad7b4f6ab8c0/dev/traffic_router/conf/server.xml#L21 That is because, aside from `ROOT.war`, the Traffic Router RPM includes an entire `lib` directory (which duplicates the `lib` directory contained in `ROOT.war`, which contains `traffic_router_connector-6.2.0.jar`, which contains `TomcatLifecycleListener.class`. In order for Tomcat to find `traffic_router_connector-6.2.0.jar` (and assuming the `CATALINA_BASE` environment variable is `${TC}/dev/traffic_router`, `${TC}/dev/traffic_router/lib` can be a relative symlink to `traffic_router/core/target/ROOT/WEB-INF/lib`, generated by `mvn compile`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
