[
https://issues.apache.org/jira/browse/IMPALA-7913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Gaal updated IMPALA-7913:
--------------------------------
Description:
When using docker/test-with-docker.py, Impala builds only the core Impala
binaries in the "build" phase/container. Test and benchmark executables are
built in later phases in the various test containers. These latter builds are
executed in parallel in the context of concurrently running containers. These
containers separate the running processes, but they share a common ccache
directory for efficiency's sake; this directory is managed by the ccache
processes running inside the containers.
Recently I have seen multiple ccache failures in this phase, when the
containers are hosting centos6 (test-with-docker.py launched with
--base-image=centos:6). The failures usually look like this:
{code:java}
2018-11-27 04:59:52.826775 [ 23%] Building CXX object
be/generated-sources/gen-cpp/CMakeFiles/ImpalaThrift.dir/RuntimeProfile_constants.cpp.o
2018-11-27 04:59:52.933599 [ 23%] Building CXX object
be/generated-sources/gen-cpp/CMakeFiles/ImpalaThrift.dir/RuntimeProfile_types.cpp.o
2018-11-27 04:59:53.041421 [ 23%] Building CXX object
be/generated-sources/gen-cpp/CMakeFiles/ImpalaThrift.dir/StatestoreService.cpp.o
2018-11-27 04:59:53.169103 [ 23%] Building CXX object
be/generated-sources/gen-cpp/CMakeFiles/ImpalaThrift.dir/StatestoreService_constants.cpp.o
2018-11-27 04:59:53.544171 [ 23%] Building CXX object
be/generated-sources/gen-cpp/CMakeFiles/ImpalaThrift.dir/StatestoreService_types.cpp.o
2018-11-27 04:59:53.576886 [ 23%] Building CXX object
be/generated-sources/gen-cpp/CMakeFiles/ImpalaThrift.dir/StatestoreSubscriber.cpp.o
2018-11-27 04:59:53.946981 ccache: FATAL: Failed to open
/home/impdev/.ccache/tmp/tmp.cpp_stderr.i-20181127-033918.7545: No such file or
directory
2018-11-27 04:59:53.947712 make[3]: ***
[be/generated-sources/gen-cpp/CMakeFiles/ImpalaThrift.dir/RuntimeProfile_constants.cpp.o]
Error 1
{code}
The issue seems to be caused by the cache being shared across containers, and
running an older ccache version under CentSO 6:
[https://ccache.samba.narkive.com/o4BSOjxG/shared-ccache-directory-between-docker-containers]
claims that ccache v3.1 (the one CentOS 6 pulls from EPEL by default) has a
problem generating unique filenames for its temporary files.
> test-with-docker can sometimes fail with a ccache fatal error when using
> CentOS 6 within the containers
> -------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-7913
> URL: https://issues.apache.org/jira/browse/IMPALA-7913
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 3.1.0
> Environment: test-with-docker
> Reporter: Laszlo Gaal
> Assignee: Laszlo Gaal
> Priority: Major
>
> When using docker/test-with-docker.py, Impala builds only the core Impala
> binaries in the "build" phase/container. Test and benchmark executables are
> built in later phases in the various test containers. These latter builds are
> executed in parallel in the context of concurrently running containers. These
> containers separate the running processes, but they share a common ccache
> directory for efficiency's sake; this directory is managed by the ccache
> processes running inside the containers.
> Recently I have seen multiple ccache failures in this phase, when the
> containers are hosting centos6 (test-with-docker.py launched with
> --base-image=centos:6). The failures usually look like this:
> {code:java}
> 2018-11-27 04:59:52.826775 [ 23%] Building CXX object
> be/generated-sources/gen-cpp/CMakeFiles/ImpalaThrift.dir/RuntimeProfile_constants.cpp.o
> 2018-11-27 04:59:52.933599 [ 23%] Building CXX object
> be/generated-sources/gen-cpp/CMakeFiles/ImpalaThrift.dir/RuntimeProfile_types.cpp.o
> 2018-11-27 04:59:53.041421 [ 23%] Building CXX object
> be/generated-sources/gen-cpp/CMakeFiles/ImpalaThrift.dir/StatestoreService.cpp.o
> 2018-11-27 04:59:53.169103 [ 23%] Building CXX object
> be/generated-sources/gen-cpp/CMakeFiles/ImpalaThrift.dir/StatestoreService_constants.cpp.o
> 2018-11-27 04:59:53.544171 [ 23%] Building CXX object
> be/generated-sources/gen-cpp/CMakeFiles/ImpalaThrift.dir/StatestoreService_types.cpp.o
> 2018-11-27 04:59:53.576886 [ 23%] Building CXX object
> be/generated-sources/gen-cpp/CMakeFiles/ImpalaThrift.dir/StatestoreSubscriber.cpp.o
> 2018-11-27 04:59:53.946981 ccache: FATAL: Failed to open
> /home/impdev/.ccache/tmp/tmp.cpp_stderr.i-20181127-033918.7545: No such file
> or directory
> 2018-11-27 04:59:53.947712 make[3]: ***
> [be/generated-sources/gen-cpp/CMakeFiles/ImpalaThrift.dir/RuntimeProfile_constants.cpp.o]
> Error 1
> {code}
> The issue seems to be caused by the cache being shared across containers, and
> running an older ccache version under CentSO 6:
> [https://ccache.samba.narkive.com/o4BSOjxG/shared-ccache-directory-between-docker-containers]
> claims that ccache v3.1 (the one CentOS 6 pulls from EPEL by default) has a
> problem generating unique filenames for its temporary files.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
