It would be really helpful this conversation can be applied in the description of each parameter. Currently, I think that is too short.
Best, Taewoo On Mon, Sep 12, 2016 at 2:19 PM, Jianfeng Jia <[email protected]> wrote: > Clear. Thanks. > > And Ian’s parameters works. I can have a on-disk components around 128M. > Thanks! > > > On Sep 12, 2016, at 12:50 PM, Sattam Alsubaiee <[email protected]> > wrote: > > > > This is the total memory size given for all datasets. Think of it as the > > buffer cache for all memory components of all indexes in that machine. > When > > it is exhausted, a victim dataset must be evicted and closed to have a > > space for another dataset. > > > > On Mon, Sep 12, 2016 at 12:29 PM, Jianfeng Jia <[email protected]> > > wrote: > > > >> I was a little confused, there is another configuration: > >> > >> storage.memorycomponent.globalbudget ( which I set to 4G) > >> > >> I was thinking this is the budget that every component on one partition > is > >> shared. Is that the case? > >> > >>> On Sep 12, 2016, at 12:16 PM, Sattam Alsubaiee <[email protected]> > >> wrote: > >>> > >>> The 128M is shared by all the memory components of the primary index > and > >>> all its secondary indexes across all io devices on that node. > >>> Also the in-memory components usually usually has fill factor of 75% > >> since > >>> the pages are 75% full and the remaining 25% is un-utilized. > >>> > >>> The page size that you have set 128KB looks reasonable for most cases. > >> Your > >>> best bet is to increase the value of storage.memorycomponent.numpage > to > >> a > >>> higher number. > >>> > >>> Sattam > >>> > >>> > >>> On Mon, Sep 12, 2016 at 11:33 AM, Jianfeng Jia <[email protected] > > > >>> wrote: > >>> > >>>> Dear devs, > >>>> > >>>> I’m using the `no-merge` compaction policy and find that the physical > >>>> flushed on-disk component is smaller than I was expected. > >>>> > >>>> Here are my related configurations > >>>> > >>>> <property> > >>>> <name>storage.memorycomponent.pagesize</name> > >>>> <value>128KB</value> > >>>> <description>The page size in bytes for pages allocated to memory > >>>> components. (Default = "131072" // 128KB) > >>>> </description> > >>>> </property> > >>>> > >>>> <property> > >>>> <name>storage.memorycomponent.numpages</name> > >>>> <value>1024</value> > >>>> <description>The number of pages to allocate for a memory component. > >>>> (Default = 256) > >>>> </description> > >>>> </property> > >>>> > >>>> With these two settings, I’m expecting the lsm component should be > 128M. > >>>> However, the flushed one is about 16M~ 20M. Do we have some > compression > >> for > >>>> the on-disk components? If so, it will be good. Otherwise, could > someone > >>>> help me to increase the component size? Thanks! > >>>> > >>>> Best, > >>>> > >>>> Jianfeng Jia > >>>> PhD Candidate of Computer Science > >>>> University of California, Irvine > >>>> > >>>> > >> > >> > >> > >> Best, > >> > >> Jianfeng Jia > >> PhD Candidate of Computer Science > >> University of California, Irvine > >> > >> > > > > Best, > > Jianfeng Jia > PhD Candidate of Computer Science > University of California, Irvine > >
