xiangdong Huang created IOTDB-106:
-------------------------------------

             Summary: How about add a new strategy to close data files in IoTDB 
according to the memory usage
                 Key: IOTDB-106
                 URL: https://issues.apache.org/jira/browse/IOTDB-106
             Project: Apache IoTDB
          Issue Type: Wish
            Reporter: xiangdong Huang


Different with other LSM-based systems, IoTDB will flush multiple memtable into 
one TsFile file. 

The TsFile can support append memtables one by one unless a "close file" 
operation is called.

Suppose $n$ memtables  are flushed into one un-closed TsFile (called unsealed 
file in the source code), these memtables' metadata will be hold in memory, 
which may lead to high cost of memory under some heavy workloads.

 

Now there are 3 strategies to trigger a "close" operation: (1) call the 
operation per `period_time_for_flush_in_second` seconds (configured in 
iotdb-engine.properties); (2) if the unsealed tsfile size reaches to 
`bufferwrite_file_size_threshold` (also configured in the same setting file); 
(3) user call `flush` in the iotdb-cli shell.

 

How about add a new strategy:  only allow $max$ metadata in memory. If the 
number in memory exceeds $max$, seal the file. In advance, $max$ can be 
calculated according to the available memory. 

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to