Joe McDonnell created IMPALA-12905:
--------------------------------------
Summary: Implement disk-based tuple caching
Key: IMPALA-12905
URL: https://issues.apache.org/jira/browse/IMPALA-12905
Project: IMPALA
Issue Type: Task
Components: Backend
Affects Versions: Impala 4.4.0
Reporter: Joe McDonnell
The TupleCacheNode caches tuples to be reused later for equivalent queries.
This tracks implementing a version that serializes tuples and stores them as
files on local disk.
This will have a few parts:
# There is a TupleCacheMgr that keeps track of what entries exist in the cache
and evicts entries as needed to make space for new entries. This will be
configured using startup flags to specify the directory, size, and cache
eviction policy.
# The TupleCacheNode will interact with the TupleCacheMgr to determine if the
entry is available. If it is, it reads the associated tuple cache file and
returns the RowBatches. If the entry does not exist, it reads RowBatches from
its child and stores them to a new file in the cache.
# The TupleReader / TupleWriter implement serialization / deserialization of
RowBatches to/from a local file. This uses the existing serialization used for
KRPC.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
