On 05/01/2018 14:13, Jakob Schiøtz wrote:
Hi again,
Yes, I have overlooked that - I just switched my repo to your branch and tried
to build :-)
Now I get an error when building TensorFlow. It is a 502 Bad Gateway,
indicating that some server is down somewhere. But is it not a problem that
the build process itself tried to download extra stuff in addition to the
source files listed in the .eb file? At least it makes the checksum checking
moot.
That's indeed a problem, but one that is hard to avoid with TensorFlow,
at least in a first iteration...
Once we're happy with the current approach, a new target could be to get
TensorFlow to build "offline".
One step at a time though... ;-)
regards,
Kenneth
Best regards
Jakob
............
WARNING: The lower priority option '-c opt' does not override the previous
value '--compilation_mode=opt'.
WARNING: The lower priority option '-c opt' does not override the previous
value '--compilation_mode=opt'.
____Downloading
https://github.com/bazelbuild/rules_closure/archive/4af89ef1db659eb41f110df189b67d4cf14073e1.tar.gz
via codeload.github.com: 40,240 bytes
____Downloading
https://github.com/bazelbuild/rules_closure/archive/4af89ef1db659eb41f110df189b67d4cf14073e1.tar.gz
via codeload.github.com: 205,436 bytes
____Loading package: tensorflow/tools/pip_package
____Loading package: @bazel_tools//tools/cpp
____Loading package: @local_jdk//
____Loading package: @local_config_cc//
____Loading complete. Analyzing...
ERROR:
/home/niflheim/schiotz/easybuild_experimental/sandybridge/build/TensorFlow/1.4.0/foss-2017b-Python-3.6.3/tensorflow-1.4.0/tensorflow/tools/pip_package/BUILD:139:1:
error loading package 'tensorflow': Encountered error while reading extension
file 'protobuf.bzl': no such package '@protobuf_archive//':
java.io.IOException: Error downloading
[http://mirror.bazel.build/github.com/google/protobuf/archive/b04e5cba356212e4e8c66c61bbe0c3a20537c5b9.tar.gz]
to
/tmp/eb-GpWEyg/tmpfJrPWS-bazel-build/external/protobuf_archive/b04e5cba356212e4e8c66c61bbe0c3a20537c5b9.tar.gz:
GET returned 502 Bad Gateway and referenced by
'//tensorflow/tools/pip_package:build_pip_package'.
ERROR:
/home/niflheim/schiotz/easybuild_experimental/sandybridge/build/TensorFlow/1.4.0/foss-2017b-Python-3.6.3/tensorflow-1.4.0/tensorflow/tools/pip_package/BUILD:139:1:
error loading package 'tensorflow': Encountered error while reading extension
file 'protobuf.bzl': no such package '@protobuf_archive//':
java.io.IOException: Error downloading
[http://mirror.bazel.build/github.com/google/protobuf/archive/b04e5cba356212e4e8c66c61bbe0c3a20537c5b9.tar.gz]
to
/tmp/eb-GpWEyg/tmpfJrPWS-bazel-build/external/protobuf_archive/b04e5cba356212e4e8c66c61bbe0c3a20537c5b9.tar.gz:
GET returned 502 Bad Gateway and referenced by
'//tensorflow/tools/pip_package:build_pip_package'.
ERROR: Analysis of target '//tensorflow/tools/pip_package:build_pip_package'
failed; build aborted: error loading package 'tensorflow': Encountered error
while reading extension file 'protobuf.bzl': no such package
'@protobuf_archive//': java.io.IOException: Error downloading
[http://mirror.bazel.build/github.com/google/protobuf/archive/b04e5cba356212e4e8c66c61bbe0c3a20537c5b9.tar.gz]
to
/tmp/eb-GpWEyg/tmpfJrPWS-bazel-build/external/protobuf_archive/b04e5cba356212e4e8c66c61bbe0c3a20537c5b9.tar.gz:
GET returned 502 Bad Gateway.
____Elapsed time: 6.561s
(at easybuild/tools/run.py:481 in parse_cmd_output)
== 2018-01-05 14:07:30,582 easyblock.py:2685 WARNING build failed (first 300 chars): cmd
"bazel --output_base=/tmp/eb-GpWEyg/tmpfJrPWS-bazel-build build
--compilation_mode=opt --config=opt --subcommands --verbose_failures --config=mkl
//tensorflow/tools/pip_package:build_pip_package" exited with exit code 1 and output:
............
On 5 Jan 2018, at 13:50, Kenneth Hoste <kenneth.ho...@ugent.be> wrote:
Hi Jakob,
On 05/01/2018 13:19, Jakob Schiøtz wrote:
Hi Kenneth,
Is it possible that you forgot to check in the patches
TensorFlow-1.4.0_swig-env.patch and TensorFlow-1.4.0_no-enum34.patch in your
PR? Attempting to build TensorFlow fails because it cannot find these.
The patch files are available from
https://github.com/easybuilders/easybuild-easyconfigs/pull/5318 (as mentioned
in the description of the PR).
regards,
Kenneth
Best regards
Jakob
On 4 Jan 2018, at 16:37, Jakob Schiøtz <schi...@fysik.dtu.dk> wrote:
Dear Kenneth, Pablo and Maxime,
Thanks for your feedback. Yes, I will try to see if I can build from source,
but I will focus on the foss toolchain since we use that one for our Python
here (we do not have the Intel MPI license, and the iomkl toolchain could not
built Python last time I tried).
I assume the reason for building from source is to ensure consistent library
versions etc. If that proves very difficult, could we perhaps in the interim
have builds (with a -bin suffix?) using the prebuilt wheels?
Best regards
Jakob
On 4 Jan 2018, at 15:29, Kenneth Hoste <kenneth.ho...@ugent.be> wrote:
Dear Jakob,
On 04/01/2018 10:23, Jakob Schiøtz wrote:
Hi,
I made a TensorFlow easyconfig a while ago depending on Python with the foss
toolchain; and including a variant with GPU support (PR 4904). The latter has
not yet been merged, probably because it is annoying to have something that can
only build on a machine with a GPU (it fails the sanity check otherwise, as
TensorFlow with GPU support cannot load on a machine without it).
Not being able to test this on a non-GPU system is a bit unfortunate, but
that's not a reason that it hasn't been merged yet, that's mostly due to a lack
of time from my side to get back to it...
Since I made that PR, two newer releases of TensorFlow have appeared (1.3 and
1.4). There are easyconfigs for 1.3 with the Intel tool chain. I am
considering making easyconfigs for TensorFlow 1.4 with Python-3.6.3-foss-2017b
(both with and without GPU support), but first I would like to know if anybody
else is doing this - it is my impression that somebody who actually know what
they are doing may be working on TensorFlow. :-)
I have spent quite a bit of time puzzling together an easyblock that supports
building TensorFlow from source, see [1].
It already works for non-GPU installations (see [2] for example), but it's not
entirely finished yet because:
* building from source with CUDA support does not work yet, the build fails
with strange Bazel errors...
* there are some issues when the TensorFlow easyblock is used together with
--use-ccache and the Intel compilers;
because two compiler wrappers are used, they end up calling each other resulting in a
"fork bomb" style situation...
I would really like to get it finished and have easyconfigs available for
TensorFlow 1.4 and newer where we properly build TensorFlow from source rather
than using the binary wheels...
Are you up for giving it a try, and maybe helping out with the problems
mentioned above?
regards,
Kenneth
[1] https://github.com/easybuilders/easybuild-easyblocks/pull/1287
[2] https://github.com/easybuilders/easybuild-easyconfigs/pull/5499
Best regards
Jakob
--
Jakob Schiøtz, professor, Ph.D.
Department of Physics
Technical University of Denmark
DK-2800 Kongens Lyngby, Denmark
http://www.fysik.dtu.dk/~schiotz/
--
Jakob Schiøtz, professor, Ph.D.
Department of Physics
Technical University of Denmark
DK-2800 Kongens Lyngby, Denmark
http://www.fysik.dtu.dk/~schiotz/
--
Jakob Schiøtz, professor, Ph.D.
Department of Physics
Technical University of Denmark
DK-2800 Kongens Lyngby, Denmark
http://www.fysik.dtu.dk/~schiotz/
--
Jakob Schiøtz, professor, Ph.D.
Department of Physics
Technical University of Denmark
DK-2800 Kongens Lyngby, Denmark
http://www.fysik.dtu.dk/~schiotz/
