KellenSunderland commented on a change in pull request #9918: Update mkldnn to the newest & Add clang build test with mkldnn. URL: https://github.com/apache/incubator-mxnet/pull/9918#discussion_r172146416
########## File path: tests/ci_build/Dockerfile.cpu_clang ########## @@ -19,3 +19,9 @@ RUN wget -O - http://apt.llvm.org/llvm-snapshot.gpg.key | apt-key add - && \ apt-get install -y clang-3.9 clang-5.0 && \ clang-3.9 --version && \ clang-5.0 --version + +# Add MKLML library, compatiable with Ubuntu16.04 +RUN wget --no-check-certificate -O /tmp/mklml.tgz https://github.com/intel/mkl-dnn/releases/download/v0.12/mklml_lnx_2018.0.1.20171227.tgz Review comment: So I listed the space and in fact it didn't save disk space ... which was suspicious until I saw what was going on. I was assuming you were downloading a tgz in the first RUN command, and then deleting it in the second. As docker stores the changes to the file system as a unique layer for each command this would mean we're storing an unnecessary tgz file in a redundant layer. In fact we weren't deleting the tgz file in the second RUN. I've created I'd recommend we use here https://gist.github.com/KellenSunderland/14778ec80b7bf1a3433811670932242a. In the gist there's an optimized version and an unoptimized version. Here's the space usage reports for each version: ```bash $ docker history optimized IMAGE CREATED CREATED BY SIZE COMMENT 4aeffe9e465d 6 minutes ago /bin/sh -c #(nop) ENV LD_LIBRARY_PATH=:/usr? 0B 2d48605c4d2b 6 minutes ago /bin/sh -c wget --no-check-certificate -O /t? 306MB 40a1e5139182 About an hour ago /bin/sh -c wget -O - http://apt.llvm.org/llv? 742MB a49366f71d6e About an hour ago /bin/sh -c /install/ubuntu_install_perl.sh 71.8MB 483f846d172f About an hour ago /bin/sh -c #(nop) COPY file:1d182312c7e82760? 1.01kB af21eeafe2d8 About an hour ago /bin/sh -c /install/ubuntu_install_r.sh 146MB 42a4f146b71e About an hour ago /bin/sh -c #(nop) COPY file:3ce9646ada9a9303? 1.14kB 9ad14f368022 About an hour ago /bin/sh -c /install/ubuntu_install_scala.sh 660MB 7609e2e90cf9 About an hour ago /bin/sh -c #(nop) COPY file:edf150d8fa20ff4f? 1.21kB 8436fb76fb8b About an hour ago /bin/sh -c /install/ubuntu_install_python.sh 843MB b23977a31829 About an hour ago /bin/sh -c #(nop) COPY file:239352104979d07c? 1.25kB efc666edaccf About an hour ago /bin/sh -c /install/ubuntu_install_core.sh 833MB 48529b485203 About an hour ago /bin/sh -c #(nop) COPY file:9f5c02599ecd6c52? 1.17kB 0458a4468cbc 5 weeks ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B <missing> 5 weeks ago /bin/sh -c mkdir -p /run/systemd && echo 'do? 7B <missing> 5 weeks ago /bin/sh -c sed -i 's/^#\s*\(deb.*universe\)$? 2.76kB <missing> 5 weeks ago /bin/sh -c rm -rf /var/lib/apt/lists/* 0B <missing> 5 weeks ago /bin/sh -c set -xe && echo '#!/bin/sh' > /? 745B <missing> 5 weeks ago /bin/sh -c #(nop) ADD file:a3344b835ea6fdc56? 112MB ~/Development/incubator-mxnet/tests/ci_build (dockerfiles) $ docker history unoptimized IMAGE CREATED CREATED BY SIZE COMMENT 84090a06ad50 32 seconds ago /bin/sh -c #(nop) ENV LD_LIBRARY_PATH=:/usr? 0B b7848b89a535 32 seconds ago /bin/sh -c tar -zxvf /tmp/mklml.tgz && c? 306MB 80dd38efa252 About an hour ago /bin/sh -c wget --no-check-certificate -O /t? 77.8MB 40a1e5139182 About an hour ago /bin/sh -c wget -O - http://apt.llvm.org/llv? 742MB a49366f71d6e About an hour ago /bin/sh -c /install/ubuntu_install_perl.sh 71.8MB 483f846d172f About an hour ago /bin/sh -c #(nop) COPY file:1d182312c7e82760? 1.01kB af21eeafe2d8 About an hour ago /bin/sh -c /install/ubuntu_install_r.sh 146MB 42a4f146b71e About an hour ago /bin/sh -c #(nop) COPY file:3ce9646ada9a9303? 1.14kB 9ad14f368022 About an hour ago /bin/sh -c /install/ubuntu_install_scala.sh 660MB 7609e2e90cf9 About an hour ago /bin/sh -c #(nop) COPY file:edf150d8fa20ff4f? 1.21kB 8436fb76fb8b About an hour ago /bin/sh -c /install/ubuntu_install_python.sh 843MB b23977a31829 About an hour ago /bin/sh -c #(nop) COPY file:239352104979d07c? 1.25kB efc666edaccf About an hour ago /bin/sh -c /install/ubuntu_install_core.sh 833MB 48529b485203 About an hour ago /bin/sh -c #(nop) COPY file:9f5c02599ecd6c52? 1.17kB 0458a4468cbc 5 weeks ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B <missing> 5 weeks ago /bin/sh -c mkdir -p /run/systemd && echo 'do? 7B <missing> 5 weeks ago /bin/sh -c sed -i 's/^#\s*\(deb.*universe\)$? 2.76kB <missing> 5 weeks ago /bin/sh -c rm -rf /var/lib/apt/lists/* 0B <missing> 5 weeks ago /bin/sh -c set -xe && echo '#!/bin/sh' > /? 745B <missing> 5 weeks ago /bin/sh -c #(nop) ADD file:a3344b835ea6fdc56? 112MB ~/Development/incubator-mxnet/tests/ci_build (dockerfiles) ``` Pretty small savings, but in the optimized version there's one less layer and there's 77.8MB saved. Again this is getting pretty nitpicky. Both versions are totally functional. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
