xiaokang commented on code in PR #28810:
URL: https://github.com/apache/doris/pull/28810#discussion_r1435461035
##########
be/src/common/config.cpp:
##########
@@ -1022,6 +1022,8 @@ DEFINE_Int32(inverted_index_read_buffer_size, "4096");
DEFINE_Int32(max_depth_in_bkd_tree, "32");
// index compaction
DEFINE_Bool(inverted_index_compaction_enable, "false");
+// index by RAM directory
+DEFINE_Bool(index_inverted_index_by_ram_dir_enable, "false ");
Review Comment:
inverted_index_ram_dir_enable
##########
be/src/olap/rowset/segment_v2/inverted_index_compound_directory.cpp:
##########
@@ -584,7 +598,12 @@ void DorisCompoundDirectory::priv_getFN(char* buffer,
const char* name) const {
strcat(buffer, name);
}
-DorisCompoundDirectory::~DorisCompoundDirectory() = default;
+DorisCompoundDirectory::~DorisCompoundDirectory() {
+ if (ram_directory != nullptr) {
+ ram_directory->finish();
Review Comment:
It's not reasonable to do heavy work in dtor, since any exception will cause
a coredump.
##########
be/src/olap/rowset/segment_v2/inverted_index_writer.cpp:
##########
@@ -510,14 +523,20 @@ class InvertedIndexColumnWriterImpl : public
InvertedIndexColumnWriter {
_directory + "/" + _segment_file_name,
_index_meta->index_id(),
_index_meta->get_index_suffix());
dir = DorisCompoundDirectory::getDirectory(_fs,
index_path.c_str(), true);
- write_null_bitmap(null_bitmap_out, dir);
+ lucene::store::Directory* bkd_dir = nullptr;
+ if (config::index_inverted_index_by_ram_dir_enable) {
+ bkd_dir =
((DorisCompoundDirectory*)dir)->getDorisRAMDirectory();
Review Comment:
If dir is not used, can it be removed for performance.
##########
be/src/olap/rowset/segment_v2/inverted_index_compound_directory.cpp:
##########
@@ -116,7 +116,13 @@ void
DorisCompoundFileWriter::sort_files(std::vector<FileInfo>& file_infos) {
void DorisCompoundFileWriter::writeCompoundFile() {
// list files in current dir
std::vector<std::string> files;
- directory->list(&files);
Review Comment:
suggest a more modular way to impl: encapsulate the ram_dir logic in
list/fileLength/... functions, instead of exposing them to caller's code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
