Clear. Thanks. And Ian’s parameters works. I can have a on-disk components around 128M. Thanks!
> On Sep 12, 2016, at 12:50 PM, Sattam Alsubaiee <[email protected]> wrote: > > This is the total memory size given for all datasets. Think of it as the > buffer cache for all memory components of all indexes in that machine. When > it is exhausted, a victim dataset must be evicted and closed to have a > space for another dataset. > > On Mon, Sep 12, 2016 at 12:29 PM, Jianfeng Jia <[email protected]> > wrote: > >> I was a little confused, there is another configuration: >> >> storage.memorycomponent.globalbudget ( which I set to 4G) >> >> I was thinking this is the budget that every component on one partition is >> shared. Is that the case? >> >>> On Sep 12, 2016, at 12:16 PM, Sattam Alsubaiee <[email protected]> >> wrote: >>> >>> The 128M is shared by all the memory components of the primary index and >>> all its secondary indexes across all io devices on that node. >>> Also the in-memory components usually usually has fill factor of 75% >> since >>> the pages are 75% full and the remaining 25% is un-utilized. >>> >>> The page size that you have set 128KB looks reasonable for most cases. >> Your >>> best bet is to increase the value of storage.memorycomponent.numpage to >> a >>> higher number. >>> >>> Sattam >>> >>> >>> On Mon, Sep 12, 2016 at 11:33 AM, Jianfeng Jia <[email protected]> >>> wrote: >>> >>>> Dear devs, >>>> >>>> I’m using the `no-merge` compaction policy and find that the physical >>>> flushed on-disk component is smaller than I was expected. >>>> >>>> Here are my related configurations >>>> >>>> <property> >>>> <name>storage.memorycomponent.pagesize</name> >>>> <value>128KB</value> >>>> <description>The page size in bytes for pages allocated to memory >>>> components. (Default = "131072" // 128KB) >>>> </description> >>>> </property> >>>> >>>> <property> >>>> <name>storage.memorycomponent.numpages</name> >>>> <value>1024</value> >>>> <description>The number of pages to allocate for a memory component. >>>> (Default = 256) >>>> </description> >>>> </property> >>>> >>>> With these two settings, I’m expecting the lsm component should be 128M. >>>> However, the flushed one is about 16M~ 20M. Do we have some compression >> for >>>> the on-disk components? If so, it will be good. Otherwise, could someone >>>> help me to increase the component size? Thanks! >>>> >>>> Best, >>>> >>>> Jianfeng Jia >>>> PhD Candidate of Computer Science >>>> University of California, Irvine >>>> >>>> >> >> >> >> Best, >> >> Jianfeng Jia >> PhD Candidate of Computer Science >> University of California, Irvine >> >> Best, Jianfeng Jia PhD Candidate of Computer Science University of California, Irvine
