Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15248
to look at the new patch set (#5).
Change subject: IMPALA-9373: more tactical IWYU fixes
......................................................................
IMPALA-9373: more tactical IWYU fixes
This is a grab-bag of fixes that I did with a mix of manual
inspection. The techniques used were:
* Getting preprocessor output for a few files by modifying
command lines from compiler_commands.json to include -E.
This is revealing because you see all the random unrelated
cruft that gets pulled in. A useful one liner to extract
an (approximate) list of headers from preprocessor output is:
grep '^#.*h' be/src/util/CMakeFiles/Util.dir/os-info.cc.i | \
grep -o '".*"' | sort -u
* Looking at the IWYU recommendations for guidance on what
headers can be removed (and what need to be added).
* Grepping for includes of headers, especially in other headers
where they become viral. An example one-liner to find these:
git grep -l 'include.*<iostream>' | grep '\.h$'
Non-exhaustive list of changes made:
-----------------------------------
Unnest classes from TmpFileMgr so we can forward-declare them.
This lets us remove tmp-file-mgr.h from buffer-pool.h and
query-state.h, which are both widely included headers in the
codebase.
Also remove webserver.h from other headers, since it
pulls in openssl-util.h and consequently a lot of
openssl headers.
Avoid including runtime/multi-precision.h in other headers.
It pulls in a lot of boost multiprecision headers that
are only needed for internal implementations of math
and decimal operations. This required replacing some
references to int128_t with __int128_t, which I don't
think significantly hurts code readability.
Also remove references to decimal-util.h where they're
not needed, since it transitively pulls in
multi-precision.h
Reduce includes of boost/date_time modules, which are
transitively many places via timestamp-value.h.
Remove transitive dependencies of timestamp-value.h
to avoid pulling in remaining boost date_time headers
where not needed. Dependent headers are:
scalar-expr-evaluator.h, expr-value.h
Remove references to debug-util.h in other headers,
because it pulls in a lot of thread headers.
Remove references to llvm-codegen.h where possible,
because it pulls in many llvm headers.
Other opportunities:
--------------------
* boost/algorithm/string.hpp includes many string algorithms
and pulls in a lot of headers.
* util/string-parser.h is a giant header with many dependencies.
* There's lots of redundancy between boost and standard c++
headers. Both pull in vast numbers of utility headers for
C++ metaprogramming and similar things. If we reduced virality
of boost headers this would help a lot, and also if we switch
to equivalent standard headers where possible (e.g. unordered_map,
unordered_set, function, bind, etc).
Compile time with clang/ASAN:
-----------------------------
Before:
real 9m6.311s
user 62m25.006s
sys 2m44.798s
After:
real 8m17.073s
user 55m38.425s
sys 2m25.808s
Change-Id: I8de71866bdf3211e53560d9bfe930e7657c4d7f1
---
M be/src/benchmarks/atod-benchmark.cc
M be/src/benchmarks/bloom-filter-benchmark.cc
M be/src/benchmarks/overflow-benchmark.cc
M be/src/codegen/codegen-anyval.cc
M be/src/codegen/codegen-anyval.h
M be/src/codegen/llvm-codegen.cc
M be/src/common/init.cc
M be/src/common/logging.cc
M be/src/common/logging.h
M be/src/common/status.cc
M be/src/common/thread-debug-info-test.cc
M be/src/common/thread-debug-info.h
M be/src/exec/aggregator.cc
M be/src/exec/blocking-plan-root-sink.cc
M be/src/exec/buffered-plan-root-sink.cc
M be/src/exec/catalog-op-executor.cc
M be/src/exec/data-sink.cc
M be/src/exec/data-sink.h
M be/src/exec/exec-node.cc
M be/src/exec/exec-node.h
M be/src/exec/filter-context.cc
M be/src/exec/filter-context.h
M be/src/exec/grouping-aggregator.cc
M be/src/exec/hash-table-test.cc
M be/src/exec/hdfs-avro-scanner-ir.cc
M be/src/exec/hdfs-orc-scanner.cc
M be/src/exec/hdfs-scan-node.cc
M be/src/exec/hdfs-scanner.cc
M be/src/exec/hdfs-sequence-scanner.cc
M be/src/exec/hdfs-text-scanner.cc
M be/src/exec/join-builder.cc
M be/src/exec/kudu-scan-node.cc
M be/src/exec/kudu-scanner.cc
M be/src/exec/kudu-table-sink.cc
M be/src/exec/kudu-table-sink.h
M be/src/exec/orc-column-readers.cc
M be/src/exec/parquet/hdfs-parquet-scanner.cc
M be/src/exec/parquet/hdfs-parquet-scanner.h
M be/src/exec/parquet/parquet-column-chunk-reader.cc
M be/src/exec/parquet/parquet-column-chunk-reader.h
M be/src/exec/parquet/parquet-column-readers.cc
M be/src/exec/parquet/parquet-common.h
M be/src/exec/parquet/parquet-version-test.cc
M be/src/exec/partitioned-hash-join-builder-ir.cc
M be/src/exec/partitioned-hash-join-builder.cc
M be/src/exec/plan-root-sink.cc
M be/src/exec/read-write-util-test.cc
M be/src/exec/row-batch-cache.h
M be/src/exec/row-batch-list-test.cc
M be/src/exec/scan-node.cc
M be/src/exec/topn-node.cc
M be/src/exec/zigzag-test.cc
M be/src/exprs/agg-fn-evaluator.cc
M be/src/exprs/agg-fn.h
M be/src/exprs/aggregate-functions-ir.cc
M be/src/exprs/aggregate-functions-test.cc
M be/src/exprs/anyval-util.cc
M be/src/exprs/anyval-util.h
M be/src/exprs/decimal-functions-ir.cc
M be/src/exprs/decimal-operators-ir.cc
M be/src/exprs/expr-test.cc
M be/src/exprs/expr-value.h
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/literal.cc
M be/src/exprs/math-functions-ir.cc
M be/src/exprs/scalar-expr-evaluator.h
M be/src/exprs/timestamp-functions-ir.cc
M be/src/exprs/timezone_db.cc
M be/src/exprs/udf-builtins.cc
M be/src/rpc/thrift-util-test.cc
M be/src/runtime/CMakeLists.txt
M be/src/runtime/bufferpool/buffer-allocator.h
M be/src/runtime/bufferpool/buffer-pool-internal.h
M be/src/runtime/bufferpool/buffer-pool-test.cc
M be/src/runtime/bufferpool/buffer-pool.cc
M be/src/runtime/bufferpool/buffer-pool.h
M be/src/runtime/client-cache.h
M be/src/runtime/coordinator-backend-state.cc
M be/src/runtime/datetime-iso-sql-format-tokenizer.cc
M be/src/runtime/datetime-parser-common.cc
M be/src/runtime/datetime-parser-common.h
M be/src/runtime/decimal-test.cc
M be/src/runtime/decimal-value.h
M be/src/runtime/decimal-value.inline.h
M be/src/runtime/descriptors.h
M be/src/runtime/exec-env.cc
M be/src/runtime/io/data-cache-test.cc
M be/src/runtime/io/disk-io-mgr-test.cc
M be/src/runtime/mem-tracker.h
D be/src/runtime/multi-precision.cc
M be/src/runtime/multi-precision.h
M be/src/runtime/query-state.cc
M be/src/runtime/query-state.h
M be/src/runtime/runtime-filter-bank.cc
M be/src/runtime/sorted-run-merger.cc
M be/src/runtime/sorted-run-merger.h
M be/src/runtime/sorter.cc
M be/src/runtime/sorter.h
M be/src/runtime/test-env.cc
M be/src/runtime/test-env.h
M be/src/runtime/timestamp-parse-util.cc
M be/src/runtime/timestamp-test.cc
M be/src/runtime/timestamp-value.h
M be/src/runtime/tmp-file-mgr-internal.h
M be/src/runtime/tmp-file-mgr-test.cc
M be/src/runtime/tmp-file-mgr.cc
M be/src/runtime/tmp-file-mgr.h
M be/src/runtime/tuple.h
M be/src/service/client-request-state-map.cc
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/control-service.h
M be/src/service/impala-beeswax-server.cc
M be/src/service/impala-http-handler.cc
M be/src/service/impala-internal-service.cc
M be/src/service/impala-server.h
M be/src/statestore/statestore.h
M be/src/udf/uda-test.cc
M be/src/util/CMakeLists.txt
M be/src/util/arithmetic-util.h
M be/src/util/auth-util.cc
M be/src/util/auth-util.h
M be/src/util/bit-stream-utils-test.cc
M be/src/util/bit-util-test.cc
M be/src/util/bit-util.h
M be/src/util/bitmap-test.cc
M be/src/util/bloom-filter.h
M be/src/util/cgroup-util.cc
M be/src/util/codec.cc
M be/src/util/codec.h
M be/src/util/debug-util.h
A be/src/util/decimal-constants.h
D be/src/util/decimal-util.cc
M be/src/util/decimal-util.h
M be/src/util/decompress.cc
M be/src/util/decompress.h
M be/src/util/dict-test.cc
M be/src/util/event-metrics.h
M be/src/util/logging-support-test.cc
M be/src/util/mem-info.cc
M be/src/util/memory-metrics.h
M be/src/util/metrics.h
M be/src/util/os-info.cc
M be/src/util/pretty-printer.h
M be/src/util/proc-info-test.cc
M be/src/util/process-state-info.cc
M be/src/util/redactor-test-utils.h
M be/src/util/runtime-profile.cc
M be/src/util/string-parser-test.cc
M be/src/util/string-parser.h
M be/src/util/symbols-util-test.cc
M be/src/util/system-state-info.cc
M be/src/util/tuple-row-compare.h
M be/src/util/uid-util-test.cc
M be/src/util/uid-util.cc
M be/src/util/uid-util.h
156 files changed, 1,081 insertions(+), 913 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/15248/5
--
To view, visit http://gerrit.cloudera.org:8080/15248
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8de71866bdf3211e53560d9bfe930e7657c4d7f1
Gerrit-Change-Number: 15248
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
