Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15914
to look at the new patch set (#3).
Change subject: IMPALA-9435: Usability enhancements for data cache access trace
......................................................................
IMPALA-9435: Usability enhancements for data cache access trace
The data cache access trace was added in IMPALA-8542 as a way
to capture a workload's cache accesses to allow later analysis.
This modifies the data cache access trace to improve usability:
1. The access trace now uses a SimpleLogger to limit the total
number of trace entries per file and total number of trace
files. This caps the disk usage for the access trace. The
behavior is controlled by the data_cache_trace_dir,
max_data_cache_trace_file_size, and max_data_cache_trace_files
startup parameters.
2. This introduces the data_cache_trace_percentage, which allows
tracing only a subset of the entries produced. It traces
accesses for a consistent subset of the cache (i.e. accesses
for a filename/mtime/offset are either always traced or
never traced). This allows for better analysis than a random
sample. Tracing a subset of accesses can reduce any performance
overhead from tracing. It also provides a way to trace a longer
time period in the same number of entries.
This also implements the ability to replay traces against a
specific cache configuration. The replayer can produce JSON output
with cache hit/miss information for the original trace and the
replay. This provides a building block for building analysis
comparing different cache sizes or cache eviction policies.
Testing:
- New backend tests in data-cache-test, data-cache-trace-test
- Manually testing the data-cache-trace-replayer
Change-Id: I0f84204d8e5145f5fa8d4851d9c19ac317db168e
---
M be/src/runtime/io/CMakeLists.txt
M be/src/runtime/io/data-cache-test.cc
A be/src/runtime/io/data-cache-trace-replayer.cc
A be/src/runtime/io/data-cache-trace-test.cc
A be/src/runtime/io/data-cache-trace.cc
A be/src/runtime/io/data-cache-trace.h
M be/src/runtime/io/data-cache.cc
M be/src/runtime/io/data-cache.h
M bin/start-impala-cluster.py
9 files changed, 1,473 insertions(+), 271 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/15914/3
--
To view, visit http://gerrit.cloudera.org:8080/15914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0f84204d8e5145f5fa8d4851d9c19ac317db168e
Gerrit-Change-Number: 15914
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <joemcdonn...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
