Nice!
> On Oct 13, 2016, at 2:56 PM, Taewoo Kim <wangs...@gmail.com> wrote:
> 
> The explanation changes for the two parameters have been merged into the
> master.
> 
> https://asterix-gerrit.ics.uci.edu/#/c/1281/3/asterixdb/asterix-installer/src/main/resources/conf/asterix-configuration.xml
> 
> Best,
> Taewoo
> 
> On Mon, Sep 12, 2016 at 5:02 PM, Taewoo Kim <wangs...@gmail.com> wrote:
> 
>> Thanks to Sattam, here is the revised version. Feel free to revise this. I
>> will upload a patch set after some revision is done.
>> 
>> *storage.memorycomponent.numpages*
>> 
>> The number of pages to allocate for a memory component. (Default = 256)
>> This budget is shared by all the memory components of the primary index
>> and all its secondary indexes across all I/O devices on a node.
>> Note: in-memory components usually has fill factor of 75% since the pages
>> are 75% full and the remaining 25% is un-utilized.
>> 
>> 
>> *storage.memorycomponent.globalbudget*
>> 
>> [4GB + 100MB] The total size of memory in bytes that the sum of all open
>> memory components cannot exceed. (Default = 512MB)
>> Consider this as the buffer cache for all memory components of all indexes
>> in a node.
>> When this budget is fully used, a victim dataset will be chosen. It must
>> be evicted and closed to make a space for another dataset.
>> 
>> 
>> Best,
>> Taewoo
>> 
>> On Mon, Sep 12, 2016 at 4:10 PM, Mike Carey <dtab...@gmail.com> wrote:
>> 
>>> +1
>>> 
>>> 
>>> 
>>> On 9/12/16 3:42 PM, Taewoo Kim wrote:
>>> 
>>>> It would be really helpful this conversation can be applied in the
>>>> description of each parameter. Currently, I think that is too short.
>>>> 
>>>> Best,
>>>> Taewoo
>>>> 
>>>> On Mon, Sep 12, 2016 at 2:19 PM, Jianfeng Jia <jianfeng....@gmail.com>
>>>> wrote:
>>>> 
>>>> Clear. Thanks.
>>>>> 
>>>>> And Ian’s parameters works. I can have a on-disk components around 128M.
>>>>> Thanks!
>>>>> 
>>>>> On Sep 12, 2016, at 12:50 PM, Sattam Alsubaiee <salsuba...@gmail.com>
>>>>>> 
>>>>> wrote:
>>>>> 
>>>>>> This is the total memory size given for all datasets. Think of it as
>>>>>> the
>>>>>> buffer cache for all memory components of all indexes in that machine.
>>>>>> 
>>>>> When
>>>>> 
>>>>>> it is exhausted, a victim dataset must be evicted and closed to have a
>>>>>> space for another dataset.
>>>>>> 
>>>>>> On Mon, Sep 12, 2016 at 12:29 PM, Jianfeng Jia <jianfeng....@gmail.com
>>>>>>> 
>>>>>> wrote:
>>>>>> 
>>>>>> I was a little confused, there is another configuration:
>>>>>>> 
>>>>>>> storage.memorycomponent.globalbudget ( which I set to 4G)
>>>>>>> 
>>>>>>> I was thinking this is the budget that every component on one
>>>>>>> partition
>>>>>>> 
>>>>>> is
>>>>> 
>>>>>> shared. Is that the case?
>>>>>>> 
>>>>>>> On Sep 12, 2016, at 12:16 PM, Sattam Alsubaiee <salsuba...@gmail.com>
>>>>>>>> 
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> The 128M is shared by all the memory components of the primary index
>>>>>>>> 
>>>>>>> and
>>>>> 
>>>>>> all its secondary indexes across all io devices on that node.
>>>>>>>> Also the in-memory components usually usually has fill factor of 75%
>>>>>>>> 
>>>>>>> since
>>>>>>> 
>>>>>>>> the pages are 75% full and the remaining 25% is un-utilized.
>>>>>>>> 
>>>>>>>> The page size that you have set 128KB looks reasonable for most
>>>>>>>> cases.
>>>>>>>> 
>>>>>>> Your
>>>>>>> 
>>>>>>>> best bet is to increase the value of storage.memorycomponent.numpage
>>>>>>>> 
>>>>>>> to
>>>>> 
>>>>>> a
>>>>>>> 
>>>>>>>> higher number.
>>>>>>>> 
>>>>>>>> Sattam
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Mon, Sep 12, 2016 at 11:33 AM, Jianfeng Jia <
>>>>>>>> jianfeng....@gmail.com
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> Dear devs,
>>>>>>>>> 
>>>>>>>>> I’m using the `no-merge` compaction policy and find that the
>>>>>>>>> physical
>>>>>>>>> flushed on-disk component is smaller than I was expected.
>>>>>>>>> 
>>>>>>>>> Here are my related configurations
>>>>>>>>> 
>>>>>>>>> <property>
>>>>>>>>>   <name>storage.memorycomponent.pagesize</name>
>>>>>>>>>   <value>128KB</value>
>>>>>>>>>   <description>The page size in bytes for pages allocated to memory
>>>>>>>>>     components. (Default = "131072" // 128KB)
>>>>>>>>>   </description>
>>>>>>>>> </property>
>>>>>>>>> 
>>>>>>>>> <property>
>>>>>>>>>   <name>storage.memorycomponent.numpages</name>
>>>>>>>>>   <value>1024</value>
>>>>>>>>>   <description>The number of pages to allocate for a memory
>>>>>>>>> component.
>>>>>>>>>     (Default = 256)
>>>>>>>>>   </description>
>>>>>>>>> </property>
>>>>>>>>> 
>>>>>>>>> With these two settings, I’m expecting the lsm component should be
>>>>>>>>> 
>>>>>>>> 128M.
>>>>> 
>>>>>> However, the flushed one is about 16M~ 20M. Do we have some
>>>>>>>>> 
>>>>>>>> compression
>>>>> 
>>>>>> for
>>>>>>> 
>>>>>>>> the on-disk components? If so, it will be good. Otherwise, could
>>>>>>>>> 
>>>>>>>> someone
>>>>> 
>>>>>> help me to increase the component size? Thanks!
>>>>>>>>> 
>>>>>>>>> Best,
>>>>>>>>> 
>>>>>>>>> Jianfeng Jia
>>>>>>>>> PhD Candidate of Computer Science
>>>>>>>>> University of California, Irvine
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>>>> Best,
>>>>>>> 
>>>>>>> Jianfeng Jia
>>>>>>> PhD Candidate of Computer Science
>>>>>>> University of California, Irvine
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>> 
>>>>> Best,
>>>>> 
>>>>> Jianfeng Jia
>>>>> PhD Candidate of Computer Science
>>>>> University of California, Irvine
>>>>> 
>>>>> 
>>>>> 
>>> 
>> 

Reply via email to