Re: [PR] [feture](inverted index) Adding Storage Format V3 for Inverted Index [doris]
doris-robot commented on PR #44414: URL: https://github.com/apache/doris/pull/44414#issuecomment-2507500657 TeamCity be ut coverage result: Function Coverage: 38.32% (9975/26029) Line Coverage: 29.41% (83535/284076) Region Coverage: 28.51% (42961/150663) Branch Coverage: 25.14% (21819/86796) Coverage Report: http://coverage.selectdb-in.cc/coverage/b2545f800e5673ebac45f4cf759ab8625bd93c21_b2545f800e5673ebac45f4cf759ab8625bd93c21/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feture](inverted index) Adding Storage Format V3 for Inverted Index [doris]
zzzxl1993 commented on PR #44414: URL: https://github.com/apache/doris/pull/44414#issuecomment-2507308904 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feture](inverted index) Adding Storage Format V3 for Inverted Index [doris]
doris-robot commented on PR #44414: URL: https://github.com/apache/doris/pull/44414#issuecomment-2505698616 TeamCity be ut coverage result: Function Coverage: 38.32% (9974/26026) Line Coverage: 29.43% (83538/283818) Region Coverage: 28.57% (42990/150463) Branch Coverage: 25.19% (21835/86694) Coverage Report: http://coverage.selectdb-in.cc/coverage/9b298b49c83fdfd77452b7295ef16d5b7d465458_9b298b49c83fdfd77452b7295ef16d5b7d465458/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feture](inverted index) Adding Storage Format V3 for Inverted Index [doris]
zzzxl1993 commented on PR #44414: URL: https://github.com/apache/doris/pull/44414#issuecomment-2505482052 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feture](inverted index) Adding Storage Format V3 for Inverted Index [doris]
zzzxl1993 commented on PR #44414: URL: https://github.com/apache/doris/pull/44414#issuecomment-2505327695 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feture](inverted index) Adding Storage Format V3 for Inverted Index [doris]
zzzxl1993 commented on PR #44414: URL: https://github.com/apache/doris/pull/44414#issuecomment-2505229716 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feture](inverted index) Adding Storage Format V3 for Inverted Index [doris]
csun5285 commented on code in PR #44414:
URL: https://github.com/apache/doris/pull/44414#discussion_r1859728400
##
be/src/olap/rowset/segment_v2/inverted_index_file_reader.cpp:
##
@@ -30,7 +30,7 @@ namespace doris::segment_v2 {
Status InvertedIndexFileReader::init(int32_t read_buffer_size) {
if (!_inited) {
_read_buffer_size = read_buffer_size;
-if (_storage_format == InvertedIndexStorageFormatPB::V2) {
+if (_storage_format >= InvertedIndexStorageFormatPB::V2) {
auto st = _init_from_v2(read_buffer_size);
Review Comment:
_init_from_file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feture](inverted index) Adding Storage Format V3 for Inverted Index [doris]
zzzxl1993 commented on PR #44414: URL: https://github.com/apache/doris/pull/44414#issuecomment-2499513179 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feture](inverted index) Adding Storage Format V3 for Inverted Index [doris]
zzzxl1993 commented on code in PR #44414:
URL: https://github.com/apache/doris/pull/44414#discussion_r1856769253
##
gensrc/proto/olap_file.proto:
##
@@ -343,6 +343,7 @@ enum IndexType {
enum InvertedIndexStorageFormatPB {
V1 = 0;
V2 = 1;
+V3 = 3;
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feture](inverted index) Adding Storage Format V3 for Inverted Index [doris]
zzzxl1993 commented on PR #44414: URL: https://github.com/apache/doris/pull/44414#issuecomment-2498259513 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feture](inverted index) Adding Storage Format V3 for Inverted Index [doris]
airborne12 commented on code in PR #44414:
URL: https://github.com/apache/doris/pull/44414#discussion_r1855684598
##
gensrc/proto/olap_file.proto:
##
@@ -343,6 +343,7 @@ enum IndexType {
enum InvertedIndexStorageFormatPB {
V1 = 0;
V2 = 1;
+V3 = 3;
Review Comment:
why 3 not 2?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feture](inverted index) Adding Storage Format V3 for Inverted Index [doris]
airborne12 commented on code in PR #44414:
URL: https://github.com/apache/doris/pull/44414#discussion_r1855680801
##
be/src/olap/rowset/beta_rowset.cpp:
##
@@ -703,10 +703,24 @@ Status
BetaRowset::show_nested_index_file(rapidjson::Value* rowset_value,
rapidjson::Document::AllocatorType&
allocator) {
const auto& fs = _rowset_meta->fs();
auto storage_format = _schema->get_inverted_index_storage_format();
-auto format_str = storage_format == InvertedIndexStorageFormatPB::V1 ?
"V1" : "V2";
+std::string format_str;
+switch (storage_format) {
+case InvertedIndexStorageFormatPB::V1:
+format_str = "V1";
+break;
+case InvertedIndexStorageFormatPB::V2:
+format_str = "V2";
+break;
+case InvertedIndexStorageFormatPB::V3:
+format_str = "V3";
+break;
+default:
+format_str = "Unknown";
Review Comment:
return Error status here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feture](inverted index) Adding Storage Format V3 for Inverted Index [doris]
airborne12 commented on code in PR #44414:
URL: https://github.com/apache/doris/pull/44414#discussion_r1855684188
##
fe/fe-core/src/main/java/org/apache/doris/cloud/datasource/CloudInternalCatalog.java:
##
@@ -354,8 +354,10 @@ public OlapFile.TabletMetaCloudPB.Builder
createTabletMetaBuilder(long tableId,
if (invertedIndexFileStorageFormat != null) {
if (invertedIndexFileStorageFormat ==
TInvertedIndexFileStorageFormat.V1) {
schemaBuilder.setInvertedIndexStorageFormat(OlapFile.InvertedIndexStorageFormatPB.V1);
-} else {
+} else if (invertedIndexFileStorageFormat ==
TInvertedIndexFileStorageFormat.V2) {
schemaBuilder.setInvertedIndexStorageFormat(OlapFile.InvertedIndexStorageFormatPB.V2);
+} else {
Review Comment:
this makes default storage format to V3?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feture](inverted index) Adding Storage Format V3 for Inverted Index [doris]
airborne12 commented on code in PR #44414:
URL: https://github.com/apache/doris/pull/44414#discussion_r1855682309
##
be/src/olap/rowset/segment_v2/inverted_index_file_writer.cpp:
##
@@ -496,7 +496,7 @@ InvertedIndexFileWriter::create_output_stream_v2() {
void
InvertedIndexFileWriter::write_version_and_indices_count(lucene::store::IndexOutput*
output) {
// Write the version number
-output->writeInt(InvertedIndexStorageFormatPB::V2);
+output->writeInt(_storage_format);
Review Comment:
may be we can change function name from create_output_stream_v2 to
create_output_stream now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feture](inverted index) Adding Storage Format V3 for Inverted Index [doris]
github-actions[bot] commented on code in PR #44414: URL: https://github.com/apache/doris/pull/44414#discussion_r1851732524 ## be/src/olap/rowset/segment_v2/inverted_index/char_filter/char_replace_char_filter.h: ## @@ -17,7 +17,7 @@ #pragma once -#include +#include // IWYU pragma: keep Review Comment: warning: 'CLucene.h' file not found [clang-diagnostic-error] ```cpp #include // IWYU pragma: keep ^ ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feture](inverted index) Adding Storage Format V3 for Inverted Index [doris]
doris-robot commented on PR #44414: URL: https://github.com/apache/doris/pull/44414#issuecomment-2490628419 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
