@David Alves So it is not working if i set memory limit as 4G but set 10G as block cache mb?
2017-04-25 2:45 GMT+09:00 David Alves <[email protected]>: > Hi Jason > > memory_limit_hard_bytes refers to all the memory consumed by the tablet > server. It has an effect in many things but arguably the biggest effect is > that the tablet server will reject all writes when memory consumption > reaches this limit. > block_cache_capacity_mb dictates how much memory should be saved for the > block cache. The block cache caches blocks that are read from disk, thus > speeding up scans that hit cached data. The block cache also has some > effect on writes, by allowing to cache bloomfilter blocks, which we lookup > to make sure that rows are unique. The page cache caches previously fetched > disk pages, but is managed by the OS and thus out of Kudu's control. > > HTH > David > > On Mon, Apr 24, 2017 at 8:24 AM, Jason Heo <[email protected]> > wrote: > >> Hello. >> >> I'm using Apache Kudu 1.3.0. >> >> I found some explanation in manual and cdh about memory_limit_hard and >> block_cache_capacity But I couldn't understand what are they exactly used >> for, what is stored at block_cache, and what's the meaning of hard limit of >> memory? >> >> In Kudu Configuration Reference: >> >> - *memory_limit_hard_bytes >> >> <https://kudu.apache.org/docs/configuration_reference.html#kudu-tserver_memory_limit_hard_bytes>*: >> Maximum amount of memory this daemon should use, in bytes. A value of 0 >> autosizes based on the total system memory. A value of -1 disables all >> memory limiting. >> - *block_cache_capacity_mb >> >> <https://kudu.apache.org/docs/configuration_reference.html#kudu-tserver_block_cache_capacity_mb>*: >> block cache capacity in MB >> >> >> In Cloudera Manager's help: >> >> - *memory_limit_hard_bytes*: Maximum amount of memory that the Kudu >> Tablet Server will use before it starts rejecting all incoming writes. >> - *block_cache_capacity_mb*: Maximum amount of memory allocated to >> the Kudu Tablet Server's block cache. >> >> Is hard_limit related with insert/update/delete and is not related to >> performance of read? Does block_cache store tablet data? Then what's the >> relationship with page cache. >> >> After watching Comcast presentation, I was surprised because they are >> using memory_limit_hard=4G, block_cache=512MB. >> >> Regards, >> >> Jason >> > >
