Thanks David Alves!! Have a nice day! :)
2017-04-26 0:26 GMT+09:00 David Alves <[email protected]>: > Hi Jason > > Yeah that's not expected to work. > The tserver might accept that config but eventually the block cache will > consume all available memory up to 4GB and everything else will be starved > for memory. > > -david > > On Tue, Apr 25, 2017 at 3:53 AM, 기준 <[email protected]> wrote: > >> @David Alves >> >> So it is not working if i set memory limit as 4G but set 10G as block >> cache mb? >> >> 2017-04-25 2:45 GMT+09:00 David Alves <[email protected]>: >> >>> Hi Jason >>> >>> memory_limit_hard_bytes refers to all the memory consumed by the >>> tablet server. It has an effect in many things but arguably the biggest >>> effect is that the tablet server will reject all writes when memory >>> consumption reaches this limit. >>> block_cache_capacity_mb dictates how much memory should be saved for >>> the block cache. The block cache caches blocks that are read from disk, >>> thus speeding up scans that hit cached data. The block cache also has some >>> effect on writes, by allowing to cache bloomfilter blocks, which we lookup >>> to make sure that rows are unique. The page cache caches previously fetched >>> disk pages, but is managed by the OS and thus out of Kudu's control. >>> >>> HTH >>> David >>> >>> On Mon, Apr 24, 2017 at 8:24 AM, Jason Heo <[email protected]> >>> wrote: >>> >>>> Hello. >>>> >>>> I'm using Apache Kudu 1.3.0. >>>> >>>> I found some explanation in manual and cdh about memory_limit_hard and >>>> block_cache_capacity But I couldn't understand what are they exactly used >>>> for, what is stored at block_cache, and what's the meaning of hard limit of >>>> memory? >>>> >>>> In Kudu Configuration Reference: >>>> >>>> - *memory_limit_hard_bytes >>>> >>>> <https://kudu.apache.org/docs/configuration_reference.html#kudu-tserver_memory_limit_hard_bytes>*: >>>> Maximum amount of memory this daemon should use, in bytes. A value of 0 >>>> autosizes based on the total system memory. A value of -1 disables all >>>> memory limiting. >>>> - *block_cache_capacity_mb >>>> >>>> <https://kudu.apache.org/docs/configuration_reference.html#kudu-tserver_block_cache_capacity_mb>*: >>>> block cache capacity in MB >>>> >>>> >>>> In Cloudera Manager's help: >>>> >>>> - *memory_limit_hard_bytes*: Maximum amount of memory that the Kudu >>>> Tablet Server will use before it starts rejecting all incoming writes. >>>> - *block_cache_capacity_mb*: Maximum amount of memory allocated to >>>> the Kudu Tablet Server's block cache. >>>> >>>> Is hard_limit related with insert/update/delete and is not related to >>>> performance of read? Does block_cache store tablet data? Then what's the >>>> relationship with page cache. >>>> >>>> After watching Comcast presentation, I was surprised because they are >>>> using memory_limit_hard=4G, block_cache=512MB. >>>> >>>> Regards, >>>> >>>> Jason >>>> >>> >>> >> >
