imay opened a new issue #1382: Add StorageRowIterator to uniform read in storage URL: https://github.com/apache/incubator-doris/issues/1382 Currently in storage engine, there is too many interface to read data. Including followings 1. SegmentReader which is used to read data from Segment 2. ColumnData which is used to read data from SegmentGroup 3. RowsetReader which is used to read data from multiple SegmentGroup(now in be_refactor branch) 4. Reader which is used to read data from table. This class finish Aggregation/Unique operation And all of these interfaces are different, however their logical is mostly same with each other. This make our system more complex and difficult to understand. What I want to uniform all these interface to StorageRowIterator. Through this interface we can access any data in storage in row format. And it can reuse some logical, for example Merge/Union. In addition, this can make our system easy to understand. The interface may look like ``` struct StorageReadOptions { std::shared_ptr<RowCursor> lower_bound; bool include_lower_bound = true; std::shared_ptr<RowCursor> upper_bound; bool include_upper_bound = true; }; class StorageRowIterator { public: StorageRowIterator() { } virtual ~StorageRowIterator() { } virtual Status init(const StorageReadOptions& opts); virtual Status next_batch(RowBlock* block) = 0; }; ``` We can assign lower and upper bounds for a scan to eliminate scanning unused data. this issue relate to #1305
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
