[
https://issues.apache.org/jira/browse/IMPALA-9371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong updated IMPALA-9371:
----------------------------------
Description:
Impala compile times are longer than necessary because .cc files pull in a lot
of headers, especially from the standard C++ library and boost. I looked at the
preprocessor output for a couple of files, and it included ~400,000 lines of
code, almost entirely from headers.
include-what-you-use is a tool that can identify cases where header includes
can be avoided or moved from .h to .cc files to reduce the number of extraneous
headers pulled into each compile unit.
Kudu has some tooling to run IWYU in a dev environment and also in their gerrit
precommit.
I was able to run iwyu against Impala as follows:
{code}
# Clone repo and check out branch to match Impala toolchain's clang version
git clone https://github.com/include-what-you-use/include-what-you-use.git
cd include-what-you-use
git checkout clang_5.0
# Build IWYU against clang from toolchain
cmake -G "Unix Makefiles"
-DIWYU_LLVM_ROOT_PATH=/opt/Impala-Toolchain/llvm-5.0.1-p2/
-DCMAKE_CXX_FLAGS=-D_GLIBCXX_USE_CXX11_ABI=0 ..
make -j $(nproc)
IWYU_BUILD_DIR=$(pwd)
# Run IWYU on the impala codebase.
cd $IMPALA_HOME
# Do an ASAN build to force Impala to generate compile_commands.json with
compile commands suitable for clang.
./buildall.sh -noclean -notests -asan
# Placeholder for additional arguments to the iwyu tool
IWYU_ARGS=""
PATH=$IWYU_BUILD_DIR:$PATH ~/repos/include-what-you-use/iwyu_tool.py -p . -j
$(nproc) -- $IWYU_ARGS > iwyu.txt
{code}
was:
Impala compile times are longer than necessary because .cc files pull in a lot
of headers, especially from the standard C++ library and boost. I looked at the
preprocessor output for a couple of files, and it included ~400,000 lines of
code, almost entirely from headers.
include-what-you-use is a tool that can identify cases where header includes
can be avoided or moved from .h to .cc files to reduce the number of extraneous
headers pulled into each compile unit.
Kudu has some tooling to run IWYU in a dev environment and also in their gerrit
precommit. The mappings they have from private to public header files can
probably be borrowed:
https://github.com/apache/kudu/tree/master/build-support/iwyu/mappings
I was able to run iwyu against Impala as follows:
{code}
# Clone repo and check out branch to match Impala toolchain's clang version
git clone https://github.com/include-what-you-use/include-what-you-use.git
cd include-what-you-use
git checkout clang_5.0
# Build IWYU against clang from toolchain
cmake -G "Unix Makefiles"
-DIWYU_LLVM_ROOT_PATH=/opt/Impala-Toolchain/llvm-5.0.1-p2/
-DCMAKE_CXX_FLAGS=-D_GLIBCXX_USE_CXX11_ABI=0 ..
make -j $(nproc)
IWYU_BUILD_DIR=$(pwd)
# Run IWYU on the impala codebase.
cd $IMPALA_HOME
# Do an ASAN build to force Impala to generate compile_commands.json with
compile commands suitable for clang.
./buildall.sh -noclean -notests -asan
PATH=$IWYU_BUILD_DIR:$PATH ~/repos/include-what-you-use/iwyu_tool.py -p . >
iwyu.txt
{code}
> Investigate include-what-you-use to improve compile times
> ---------------------------------------------------------
>
> Key: IMPALA-9371
> URL: https://issues.apache.org/jira/browse/IMPALA-9371
> Project: IMPALA
> Issue Type: Improvement
> Components: Infrastructure
> Reporter: Tim Armstrong
> Priority: Major
>
> Impala compile times are longer than necessary because .cc files pull in a
> lot of headers, especially from the standard C++ library and boost. I looked
> at the preprocessor output for a couple of files, and it included ~400,000
> lines of code, almost entirely from headers.
> include-what-you-use is a tool that can identify cases where header includes
> can be avoided or moved from .h to .cc files to reduce the number of
> extraneous headers pulled into each compile unit.
> Kudu has some tooling to run IWYU in a dev environment and also in their
> gerrit precommit.
> I was able to run iwyu against Impala as follows:
> {code}
> # Clone repo and check out branch to match Impala toolchain's clang version
> git clone https://github.com/include-what-you-use/include-what-you-use.git
> cd include-what-you-use
> git checkout clang_5.0
> # Build IWYU against clang from toolchain
> cmake -G "Unix Makefiles"
> -DIWYU_LLVM_ROOT_PATH=/opt/Impala-Toolchain/llvm-5.0.1-p2/
> -DCMAKE_CXX_FLAGS=-D_GLIBCXX_USE_CXX11_ABI=0 ..
> make -j $(nproc)
> IWYU_BUILD_DIR=$(pwd)
> # Run IWYU on the impala codebase.
> cd $IMPALA_HOME
> # Do an ASAN build to force Impala to generate compile_commands.json with
> compile commands suitable for clang.
> ./buildall.sh -noclean -notests -asan
> # Placeholder for additional arguments to the iwyu tool
> IWYU_ARGS=""
> PATH=$IWYU_BUILD_DIR:$PATH ~/repos/include-what-you-use/iwyu_tool.py -p . -j
> $(nproc) -- $IWYU_ARGS > iwyu.txt
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
