DaweiLiu created IOTDB-509:
------------------------------
Summary: Optimize TsFileReader to reduce unnecessary GC and IO.
Key: IOTDB-509
URL: https://issues.apache.org/jira/browse/IOTDB-509
Project: Apache IoTDB
Issue Type: Wish
Components: Core/TsFile
Reporter: DaweiLiu
I think there are still two parts of TsFile that can be optimized
# Reduce unnecessary IO. The current reading is carried out according to the
Chunk level. I think we can put pageindex together. When the time in the filter
contains the chunk time, all chunk data will be read out and returned directly.
When only intersecting, we can determine which pages to read out by reading
pageindex, thus reducing unnecessary data reading
# The reduction in the gc, read the data returned is based on batchData
structure, and the amount of data that is aligned with the page each time, that
is, each time when you call next () method reads, will the new a batchData, if
the query has experienced thousands of page, that means we have the new 10000
batchData.So I think that we should isolate the data of the page. We do io and
serialization / decoding from the hard disk one page at a time, but when it is
handed over to the business, it should be a data structure that can be reused.
He is Fixed length, just like read (ByteBuffer) in JDK
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
