Hi all, Following up on the ongoing discussion around LLAP, I'd like to raise a question about our current caching approach. At the moment, we rely primarily on mmap files for data caching in Hive LLAP. Since mmap requires mapping file contents into a region of memory, the host's memory capacity directly limits how much data we can cache. As a rule of thumb, memory consumption is roughly file size × 0.06 — this is also documented in Microsoft's HDInsight sizing guide: https://learn.microsoft.com/en-us/azure/hdinsight/interactive-query/hive-llap-sizing-guide Because of this memory constraint, the number of data blocks we can cache is quite limited. Given that NVMe SSDs are now widely available, I'd like to propose evaluating a local file-cache implementation instead — for example, something along the lines of the Alluxio libraries backed by RocksDB. This would remove memory capacity as the limiting factor for cache size. Notably, this is the approach Trino has already taken with its native file cache. Would it be more cost-effective for us to move in this direction? Happy to hear your thoughts, and I'm glad to help prototype or benchmark if there's interest. Best regards, Lisoda
---- 回复的原邮件 ---- | 发件人 | Denys Kuzmenko<[email protected]> | | 日期 | 2026年09月10日 22:25 | | 收件人 | [email protected] | | 抄送至 | | | 主题 | Re: Performance evaluation: Hive 4.2.0 LLAP, Trino 483, Hive 4.2.0 on MR3 | Hi Sungwoo, We are setting up the infrastructure for performance testing and would like to understand if you have made any Hive configuration tuning on the HS2, Tez AM, or LLAP side. Could you please share any relevant Hive configuration changes or tuning parameters you used? Thank you, Denys
