Re: [PR] [Improve](columns) update columns and ut [doris]
yiguolei merged PR #42269: URL: https://github.com/apache/doris/pull/42269 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
doris-robot commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2522065994 ClickBench: Total hot run time: 32.46 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit afe066ddd5e098c9e011f3918a18343551851c6a, data reload: false query1 0.030.040.03 query2 0.070.040.03 query3 0.240.060.07 query4 1.630.100.10 query5 0.410.430.41 query6 1.130.670.65 query7 0.020.010.01 query8 0.040.030.03 query9 0.590.500.50 query10 0.540.550.55 query11 0.130.100.11 query12 0.130.110.11 query13 0.610.600.59 query14 2.722.762.71 query15 0.900.830.82 query16 0.390.390.40 query17 1.071.061.01 query18 0.230.210.22 query19 1.961.842.06 query20 0.020.010.02 query21 15.36 0.590.57 query22 2.311.932.32 query23 17.17 0.940.75 query24 2.820.921.80 query25 0.210.180.11 query26 0.400.140.13 query27 0.070.050.04 query28 10.36 1.101.08 query29 12.54 3.223.17 query30 0.240.060.06 query31 2.870.380.38 query32 3.280.470.47 query33 2.993.043.07 query34 17.16 4.484.41 query35 4.504.484.45 query36 0.680.490.50 query37 0.090.060.06 query38 0.050.040.03 query39 0.030.020.03 query40 0.160.120.12 query41 0.090.030.02 query42 0.030.020.02 query43 0.040.030.03 Total cold run time: 106.31 s Total hot run time: 32.46 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
doris-robot commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2522061586 TPC-DS: Total hot run time: 197326 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit afe066ddd5e098c9e011f3918a18343551851c6a, data reload: false query1 1494965 972 965 query2 6244214220292029 query3 10932 445543874387 query4 66540 29066 23621 23621 query5 4971474 491 474 query6 428 200 200 200 query7 5597315 302 302 query8 326 241 243 241 query9 9021263726222622 query10 458 241 241 241 query11 17315 15429 15856 15429 query12 150 100 100 100 query13 1562405 414 405 query14 10693 785969846984 query15 215 199 198 198 query16 7257462 498 462 query17 1332600 602 600 query18 1873334 327 327 query19 216 158 157 157 query20 119 109 111 109 query21 219 103 106 103 query22 4646459545384538 query23 34663 34703 35222 34703 query24 5453253624722472 query25 492 387 390 387 query26 652 153 158 153 query27 1782285 296 285 query28 4266248524522452 query29 659 451 417 417 query30 208 148 148 148 query31 1012826 851 826 query32 67 56 56 56 query33 396 288 307 288 query34 935 520 544 520 query35 891 767 749 749 query36 1088965 1002965 query37 117 80 75 75 query38 4475447244764472 query39 1544146314751463 query40 220 97 96 96 query41 44 41 43 41 query42 111 101 96 96 query43 537 480 494 480 query44 1212871 838 838 query45 198 175 175 175 query46 1192748 741 741 query47 2000195218711871 query48 410 303 353 303 query49 735 407 397 397 query50 851 401 406 401 query51 7392730770217021 query52 97 92 89 89 query53 254 187 184 184 query54 508 389 405 389 query55 80 77 78 77 query56 258 242 232 232 query57 1233115011171117 query58 213 221 224 221 query59 3235307828942894 query60 271 247 275 247 query61 108 120 106 106 query62 765 663 660 660 query63 210 188 191 188 query64 1373678 665 665 query65 3327316732043167 query66 717 300 305 300 query67 16050 15531 15550 15531 query68 4151596 593 593 query69 431 265 261 261 query70 1214114211531142 query71 366 243 246 243 query72 6236381240753812 query73 766 364 365 364 query74 10239 896390778963 query75 3373266526732665 query76 1895105210881052 query77 476 265 274 265 query78 10577 952994979497 query79 1214594 608 594 query80 865 435 441 435 query81 472 236 229 229 query82 1265120 126 120 query83 174 145 145 145 query84 281 68 76 68 query85 860 307 297 297 query86 339 309 298 298 query87 4593452145734521 query88 3021224021832183 query89 428 297 291 291 query90 2016186 187 186 query91 144 104 103 103 query92 66 49 54 49 query93 1199536 551 536 query94 776 297 287 287 query95 357 249 250 249 query96 609 273 284 273 query97 2866266726392639 query98 229 203 208 203 query99 1592132013111311 Total cold run time: 316999 ms Total hot run time: 197326 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [Improve](columns) update columns and ut [doris]
doris-robot commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2522058873 TeamCity be ut coverage result: Function Coverage: 38.52% (10015/26000) Line Coverage: 29.54% (83991/284306) Region Coverage: 28.64% (43159/150691) Branch Coverage: 25.23% (21942/86952) Coverage Report: http://coverage.selectdb-in.cc/coverage/afe066ddd5e098c9e011f3918a18343551851c6a_afe066ddd5e098c9e011f3918a18343551851c6a/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
doris-robot commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2522051093 TPC-H: Total hot run time: 40545 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit afe066ddd5e098c9e011f3918a18343551851c6a, data reload: false -- Round 1 -- q1 17594 753873517351 q2 2069180 169 169 q3 10617 112411841124 q4 10292 763 707 707 q5 7614272027862720 q6 246 147 147 147 q7 1009630 602 602 q8 9252187819471878 q9 6752655564936493 q10 7046231723462317 q11 479 264 262 262 q12 454 220 223 220 q13 17761 304130143014 q14 244 221 220 220 q15 572 542 527 527 q16 658 578 590 578 q17 990 620 650 620 q18 7361673767926737 q19 1382109910561056 q20 469 182 180 180 q21 4209335833153315 q22 379 312 308 308 Total cold run time: 107449 ms Total hot run time: 40545 ms - Round 2, with runtime_filter_mode=off - q1 7340731972727272 q2 329 228 249 228 q3 2920294229712942 q4 2109187419241874 q5 5655568057365680 q6 230 137 143 137 q7 2264184418161816 q8 3401358135633563 q9 8890916890179017 q10 3602357035723570 q11 596 503 501 501 q12 838 621 613 613 q13 12044 329132963291 q14 310 278 267 267 q15 576 525 517 517 q16 702 659 628 628 q17 1875168416501650 q18 8379773575757575 q19 1699160615211521 q20 2138192018801880 q21 5681554455045504 q22 668 603 568 568 Total cold run time: 72246 ms Total hot run time: 60614 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
amorynan commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2521924161 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
amorynan commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2521921690 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
amorynan commented on code in PR #42269:
URL: https://github.com/apache/doris/pull/42269#discussion_r1872462818
##
be/src/vec/columns/column_array.cpp:
##
@@ -417,7 +418,8 @@ void ColumnArray::pop_back(size_t n) {
void ColumnArray::reserve(size_t n) {
get_offsets().reserve(n);
get_data().reserve(
-n); /// The average size of arrays is not taken into account here.
Or it is considered to be no more than 1.
+get_offsets()
Review Comment:
drop this file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
amorynan commented on code in PR #42269:
URL: https://github.com/apache/doris/pull/42269#discussion_r1872462556
##
be/test/vec/columns/common_column_test.h:
##
@@ -0,0 +1,766 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include
+#include
+#include
+
+#include "olap/schema.h"
+#include "vec/columns/column.h"
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_map.h"
+#include "vec/columns/columns_number.h"
+#include "vec/core/field.h"
+#include "vec/core/sort_block.h"
+#include "vec/core/sort_description.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_map.h"
+
+// this test is gonna to be a column test template for all column which should
make ut test to coverage the function defined in column
+// for example column_array should test this function:
+// size, reserve, resize, empty, byte_size, allocated_bytes, clone_resized,
+// get_shrinked_column, filter, filter_by_selector, serialize_vec,
deserialize_vec, get_max_row_byte_size
+//
+namespace doris::vectorized {
+
+class CommonColumnTest : public ::testing::Test {
+public:
+void SetUp() override {
+col_str = ColumnString::create();
+col_str->insert_data("aaa", 3);
+col_str->insert_data("bb", 2);
+col_str->insert_data("", 4);
+
+col_int = ColumnInt64::create();
+col_int->insert_value(1);
+col_int->insert_value(2);
+col_int->insert_value(3);
+
+col_dcm = ColumnDecimal64::create(0, 3);
+col_dcm->insert_value(1.23);
+col_dcm->insert_value(4.56);
+col_dcm->insert_value(7.89);
+
+col_arr = ColumnArray::create(ColumnInt64::create(),
ColumnArray::ColumnOffsets::create());
+Array array1 = {1, 2, 3};
+Array array2 = {4};
+col_arr->insert(array1);
+col_arr->insert(Array());
+col_arr->insert(array2);
+
+col_map = ColumnMap::create(ColumnString::create(),
ColumnInt64::create(),
+ColumnArray::ColumnOffsets::create());
+Array k1 = {"a", "b", "c"};
+Array v1 = {1, 2, 3};
+Array k2 = {"d"};
+Array v2 = {4};
+Array a = Array();
+Map map1, map2, map3;
+map1.push_back(k1);
+map1.push_back(v1);
+col_map->insert(map1);
+map3.push_back(a);
+map3.push_back(a);
+col_map->insert(map3);
+map2.push_back(k2);
+map2.push_back(v2);
+col_map->insert(map2);
+}
+
+ColumnString::MutablePtr col_str;
+ColumnInt64::MutablePtr col_int;
+ColumnDecimal64::MutablePtr col_dcm;
+ColumnArray::MutablePtr col_arr;
+ColumnMap::MutablePtr col_map;
+
+void checkColumn(const IColumn& col1, const IColumn& col2, const
IDataType& dataType,
+ size_t column_size) {
+if (WhichDataType(dataType).is_map()) {
+auto map1 = check_and_get_column(col1);
+auto map2 = check_and_get_column(col2);
+const DataTypeMap& rhs_map = static_cast(dataType);
+checkColumn(map1->get_keys(), map2->get_keys(),
*rhs_map.get_key_type(),
+map1->get_keys().size());
+checkColumn(map2->get_values(), map2->get_values(),
*rhs_map.get_value_type(),
+map1->get_values().size());
+} else {
+if (WhichDataType(dataType).is_int8()) {
+auto c1 = check_and_get_column(col1);
+auto c2 = check_and_get_column(col2);
+for (size_t i = 0; i < column_size; ++i) {
+EXPECT_EQ(c1->get_element(i), c2->get_element(i));
+}
+} else if (WhichDataType(dataType).is_int16()) {
+auto c1 = check_and_get_column(col1);
+auto c2 = check_and_get_column(col2);
+for (size_t i = 0; i < column_size; ++i) {
+EXPECT_EQ(c1->get_element(i), c2->get_element(i));
+}
+} else if (WhichDataType(dataType).is_int32()) {
+auto c1 = check_and_get_column(col1);
+auto c2 =
Re: [PR] [Improve](columns) update columns and ut [doris]
amorynan commented on code in PR #42269:
URL: https://github.com/apache/doris/pull/42269#discussion_r1872461938
##
be/test/vec/columns/common_column_test.h:
##
@@ -0,0 +1,766 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include
+#include
+#include
+
+#include "olap/schema.h"
+#include "vec/columns/column.h"
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_map.h"
+#include "vec/columns/columns_number.h"
+#include "vec/core/field.h"
+#include "vec/core/sort_block.h"
+#include "vec/core/sort_description.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_map.h"
+
+// this test is gonna to be a column test template for all column which should
make ut test to coverage the function defined in column
+// for example column_array should test this function:
+// size, reserve, resize, empty, byte_size, allocated_bytes, clone_resized,
+// get_shrinked_column, filter, filter_by_selector, serialize_vec,
deserialize_vec, get_max_row_byte_size
+//
+namespace doris::vectorized {
+
+class CommonColumnTest : public ::testing::Test {
+public:
+void SetUp() override {
+col_str = ColumnString::create();
+col_str->insert_data("aaa", 3);
+col_str->insert_data("bb", 2);
+col_str->insert_data("", 4);
+
+col_int = ColumnInt64::create();
+col_int->insert_value(1);
+col_int->insert_value(2);
+col_int->insert_value(3);
+
+col_dcm = ColumnDecimal64::create(0, 3);
+col_dcm->insert_value(1.23);
+col_dcm->insert_value(4.56);
+col_dcm->insert_value(7.89);
+
+col_arr = ColumnArray::create(ColumnInt64::create(),
ColumnArray::ColumnOffsets::create());
+Array array1 = {1, 2, 3};
+Array array2 = {4};
+col_arr->insert(array1);
+col_arr->insert(Array());
+col_arr->insert(array2);
+
+col_map = ColumnMap::create(ColumnString::create(),
ColumnInt64::create(),
+ColumnArray::ColumnOffsets::create());
+Array k1 = {"a", "b", "c"};
+Array v1 = {1, 2, 3};
+Array k2 = {"d"};
+Array v2 = {4};
+Array a = Array();
+Map map1, map2, map3;
+map1.push_back(k1);
+map1.push_back(v1);
+col_map->insert(map1);
+map3.push_back(a);
+map3.push_back(a);
+col_map->insert(map3);
+map2.push_back(k2);
+map2.push_back(v2);
+col_map->insert(map2);
+}
+
+ColumnString::MutablePtr col_str;
+ColumnInt64::MutablePtr col_int;
+ColumnDecimal64::MutablePtr col_dcm;
+ColumnArray::MutablePtr col_arr;
+ColumnMap::MutablePtr col_map;
+
+void checkColumn(const IColumn& col1, const IColumn& col2, const
IDataType& dataType,
+ size_t column_size) {
+if (WhichDataType(dataType).is_map()) {
+auto map1 = check_and_get_column(col1);
+auto map2 = check_and_get_column(col2);
+const DataTypeMap& rhs_map = static_cast(dataType);
+checkColumn(map1->get_keys(), map2->get_keys(),
*rhs_map.get_key_type(),
+map1->get_keys().size());
+checkColumn(map2->get_values(), map2->get_values(),
*rhs_map.get_value_type(),
+map1->get_values().size());
+} else {
+if (WhichDataType(dataType).is_int8()) {
+auto c1 = check_and_get_column(col1);
+auto c2 = check_and_get_column(col2);
+for (size_t i = 0; i < column_size; ++i) {
+EXPECT_EQ(c1->get_element(i), c2->get_element(i));
+}
+} else if (WhichDataType(dataType).is_int16()) {
+auto c1 = check_and_get_column(col1);
+auto c2 = check_and_get_column(col2);
+for (size_t i = 0; i < column_size; ++i) {
+EXPECT_EQ(c1->get_element(i), c2->get_element(i));
+}
+} else if (WhichDataType(dataType).is_int32()) {
+auto c1 = check_and_get_column(col1);
+auto c2 =
Re: [PR] [Improve](columns) update columns and ut [doris]
amorynan commented on code in PR #42269:
URL: https://github.com/apache/doris/pull/42269#discussion_r1872460677
##
be/test/vec/columns/common_column_test.h:
##
@@ -0,0 +1,766 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include
+#include
+#include
+
+#include "olap/schema.h"
+#include "vec/columns/column.h"
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_map.h"
+#include "vec/columns/columns_number.h"
+#include "vec/core/field.h"
+#include "vec/core/sort_block.h"
+#include "vec/core/sort_description.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_map.h"
+
+// this test is gonna to be a column test template for all column which should
make ut test to coverage the function defined in column
+// for example column_array should test this function:
+// size, reserve, resize, empty, byte_size, allocated_bytes, clone_resized,
+// get_shrinked_column, filter, filter_by_selector, serialize_vec,
deserialize_vec, get_max_row_byte_size
+//
+namespace doris::vectorized {
+
+class CommonColumnTest : public ::testing::Test {
+public:
+void SetUp() override {
+col_str = ColumnString::create();
+col_str->insert_data("aaa", 3);
+col_str->insert_data("bb", 2);
+col_str->insert_data("", 4);
+
+col_int = ColumnInt64::create();
+col_int->insert_value(1);
+col_int->insert_value(2);
+col_int->insert_value(3);
+
+col_dcm = ColumnDecimal64::create(0, 3);
+col_dcm->insert_value(1.23);
+col_dcm->insert_value(4.56);
+col_dcm->insert_value(7.89);
+
+col_arr = ColumnArray::create(ColumnInt64::create(),
ColumnArray::ColumnOffsets::create());
+Array array1 = {1, 2, 3};
+Array array2 = {4};
+col_arr->insert(array1);
+col_arr->insert(Array());
+col_arr->insert(array2);
+
+col_map = ColumnMap::create(ColumnString::create(),
ColumnInt64::create(),
+ColumnArray::ColumnOffsets::create());
+Array k1 = {"a", "b", "c"};
+Array v1 = {1, 2, 3};
+Array k2 = {"d"};
+Array v2 = {4};
+Array a = Array();
+Map map1, map2, map3;
+map1.push_back(k1);
+map1.push_back(v1);
+col_map->insert(map1);
+map3.push_back(a);
+map3.push_back(a);
+col_map->insert(map3);
+map2.push_back(k2);
+map2.push_back(v2);
+col_map->insert(map2);
+}
+
+ColumnString::MutablePtr col_str;
+ColumnInt64::MutablePtr col_int;
+ColumnDecimal64::MutablePtr col_dcm;
+ColumnArray::MutablePtr col_arr;
+ColumnMap::MutablePtr col_map;
+
+void checkColumn(const IColumn& col1, const IColumn& col2, const
IDataType& dataType,
+ size_t column_size) {
+if (WhichDataType(dataType).is_map()) {
+auto map1 = check_and_get_column(col1);
+auto map2 = check_and_get_column(col2);
+const DataTypeMap& rhs_map = static_cast(dataType);
+checkColumn(map1->get_keys(), map2->get_keys(),
*rhs_map.get_key_type(),
+map1->get_keys().size());
+checkColumn(map2->get_values(), map2->get_values(),
*rhs_map.get_value_type(),
+map1->get_values().size());
+} else {
+if (WhichDataType(dataType).is_int8()) {
+auto c1 = check_and_get_column(col1);
+auto c2 = check_and_get_column(col2);
+for (size_t i = 0; i < column_size; ++i) {
+EXPECT_EQ(c1->get_element(i), c2->get_element(i));
+}
+} else if (WhichDataType(dataType).is_int16()) {
+auto c1 = check_and_get_column(col1);
+auto c2 = check_and_get_column(col2);
+for (size_t i = 0; i < column_size; ++i) {
+EXPECT_EQ(c1->get_element(i), c2->get_element(i));
+}
+} else if (WhichDataType(dataType).is_int32()) {
+auto c1 = check_and_get_column(col1);
+auto c2 =
Re: [PR] [Improve](columns) update columns and ut [doris]
amorynan commented on code in PR #42269:
URL: https://github.com/apache/doris/pull/42269#discussion_r1870959938
##
be/test/vec/columns/common_column_test.h:
##
@@ -0,0 +1,766 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include
+#include
+#include
+
+#include "olap/schema.h"
+#include "vec/columns/column.h"
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_map.h"
+#include "vec/columns/columns_number.h"
+#include "vec/core/field.h"
+#include "vec/core/sort_block.h"
+#include "vec/core/sort_description.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_map.h"
+
+// this test is gonna to be a column test template for all column which should
make ut test to coverage the function defined in column
+// for example column_array should test this function:
+// size, reserve, resize, empty, byte_size, allocated_bytes, clone_resized,
+// get_shrinked_column, filter, filter_by_selector, serialize_vec,
deserialize_vec, get_max_row_byte_size
+//
+namespace doris::vectorized {
+
+class CommonColumnTest : public ::testing::Test {
+public:
+void SetUp() override {
+col_str = ColumnString::create();
+col_str->insert_data("aaa", 3);
+col_str->insert_data("bb", 2);
+col_str->insert_data("", 4);
+
+col_int = ColumnInt64::create();
+col_int->insert_value(1);
+col_int->insert_value(2);
+col_int->insert_value(3);
+
+col_dcm = ColumnDecimal64::create(0, 3);
+col_dcm->insert_value(1.23);
+col_dcm->insert_value(4.56);
+col_dcm->insert_value(7.89);
+
+col_arr = ColumnArray::create(ColumnInt64::create(),
ColumnArray::ColumnOffsets::create());
+Array array1 = {1, 2, 3};
+Array array2 = {4};
+col_arr->insert(array1);
+col_arr->insert(Array());
+col_arr->insert(array2);
+
+col_map = ColumnMap::create(ColumnString::create(),
ColumnInt64::create(),
+ColumnArray::ColumnOffsets::create());
+Array k1 = {"a", "b", "c"};
+Array v1 = {1, 2, 3};
+Array k2 = {"d"};
+Array v2 = {4};
+Array a = Array();
+Map map1, map2, map3;
+map1.push_back(k1);
+map1.push_back(v1);
+col_map->insert(map1);
+map3.push_back(a);
+map3.push_back(a);
+col_map->insert(map3);
+map2.push_back(k2);
+map2.push_back(v2);
+col_map->insert(map2);
+}
+
+ColumnString::MutablePtr col_str;
+ColumnInt64::MutablePtr col_int;
+ColumnDecimal64::MutablePtr col_dcm;
+ColumnArray::MutablePtr col_arr;
+ColumnMap::MutablePtr col_map;
+
+void checkColumn(const IColumn& col1, const IColumn& col2, const
IDataType& dataType,
+ size_t column_size) {
+if (WhichDataType(dataType).is_map()) {
+auto map1 = check_and_get_column(col1);
+auto map2 = check_and_get_column(col2);
+const DataTypeMap& rhs_map = static_cast(dataType);
+checkColumn(map1->get_keys(), map2->get_keys(),
*rhs_map.get_key_type(),
+map1->get_keys().size());
+checkColumn(map2->get_values(), map2->get_values(),
*rhs_map.get_value_type(),
+map1->get_values().size());
+} else {
+if (WhichDataType(dataType).is_int8()) {
+auto c1 = check_and_get_column(col1);
+auto c2 = check_and_get_column(col2);
+for (size_t i = 0; i < column_size; ++i) {
+EXPECT_EQ(c1->get_element(i), c2->get_element(i));
+}
+} else if (WhichDataType(dataType).is_int16()) {
+auto c1 = check_and_get_column(col1);
+auto c2 = check_and_get_column(col2);
+for (size_t i = 0; i < column_size; ++i) {
+EXPECT_EQ(c1->get_element(i), c2->get_element(i));
+}
+} else if (WhichDataType(dataType).is_int32()) {
+auto c1 = check_and_get_column(col1);
+auto c2 =
Re: [PR] [Improve](columns) update columns and ut [doris]
amorynan commented on code in PR #42269:
URL: https://github.com/apache/doris/pull/42269#discussion_r1870955132
##
be/test/vec/columns/common_column_test.h:
##
@@ -0,0 +1,766 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include
+#include
+#include
+
+#include "olap/schema.h"
+#include "vec/columns/column.h"
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_map.h"
+#include "vec/columns/columns_number.h"
+#include "vec/core/field.h"
+#include "vec/core/sort_block.h"
+#include "vec/core/sort_description.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_map.h"
+
+// this test is gonna to be a column test template for all column which should
make ut test to coverage the function defined in column
+// for example column_array should test this function:
+// size, reserve, resize, empty, byte_size, allocated_bytes, clone_resized,
+// get_shrinked_column, filter, filter_by_selector, serialize_vec,
deserialize_vec, get_max_row_byte_size
+//
+namespace doris::vectorized {
+
+class CommonColumnTest : public ::testing::Test {
+public:
+void SetUp() override {
+col_str = ColumnString::create();
+col_str->insert_data("aaa", 3);
+col_str->insert_data("bb", 2);
+col_str->insert_data("", 4);
+
+col_int = ColumnInt64::create();
+col_int->insert_value(1);
+col_int->insert_value(2);
+col_int->insert_value(3);
+
+col_dcm = ColumnDecimal64::create(0, 3);
+col_dcm->insert_value(1.23);
+col_dcm->insert_value(4.56);
+col_dcm->insert_value(7.89);
+
+col_arr = ColumnArray::create(ColumnInt64::create(),
ColumnArray::ColumnOffsets::create());
+Array array1 = {1, 2, 3};
+Array array2 = {4};
+col_arr->insert(array1);
+col_arr->insert(Array());
+col_arr->insert(array2);
+
+col_map = ColumnMap::create(ColumnString::create(),
ColumnInt64::create(),
+ColumnArray::ColumnOffsets::create());
+Array k1 = {"a", "b", "c"};
+Array v1 = {1, 2, 3};
+Array k2 = {"d"};
+Array v2 = {4};
+Array a = Array();
+Map map1, map2, map3;
+map1.push_back(k1);
+map1.push_back(v1);
+col_map->insert(map1);
+map3.push_back(a);
+map3.push_back(a);
+col_map->insert(map3);
+map2.push_back(k2);
+map2.push_back(v2);
+col_map->insert(map2);
+}
+
+ColumnString::MutablePtr col_str;
+ColumnInt64::MutablePtr col_int;
+ColumnDecimal64::MutablePtr col_dcm;
+ColumnArray::MutablePtr col_arr;
+ColumnMap::MutablePtr col_map;
+
+void checkColumn(const IColumn& col1, const IColumn& col2, const
IDataType& dataType,
+ size_t column_size) {
+if (WhichDataType(dataType).is_map()) {
+auto map1 = check_and_get_column(col1);
+auto map2 = check_and_get_column(col2);
+const DataTypeMap& rhs_map = static_cast(dataType);
+checkColumn(map1->get_keys(), map2->get_keys(),
*rhs_map.get_key_type(),
+map1->get_keys().size());
+checkColumn(map2->get_values(), map2->get_values(),
*rhs_map.get_value_type(),
+map1->get_values().size());
+} else {
+if (WhichDataType(dataType).is_int8()) {
+auto c1 = check_and_get_column(col1);
+auto c2 = check_and_get_column(col2);
+for (size_t i = 0; i < column_size; ++i) {
+EXPECT_EQ(c1->get_element(i), c2->get_element(i));
+}
+} else if (WhichDataType(dataType).is_int16()) {
+auto c1 = check_and_get_column(col1);
+auto c2 = check_and_get_column(col2);
+for (size_t i = 0; i < column_size; ++i) {
+EXPECT_EQ(c1->get_element(i), c2->get_element(i));
+}
+} else if (WhichDataType(dataType).is_int32()) {
+auto c1 = check_and_get_column(col1);
+auto c2 =
Re: [PR] [Improve](columns) update columns and ut [doris]
amorynan commented on code in PR #42269:
URL: https://github.com/apache/doris/pull/42269#discussion_r1870963099
##
be/test/vec/columns/common_column_test.h:
##
@@ -0,0 +1,766 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include
+#include
+#include
+
+#include "olap/schema.h"
+#include "vec/columns/column.h"
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_map.h"
+#include "vec/columns/columns_number.h"
+#include "vec/core/field.h"
+#include "vec/core/sort_block.h"
+#include "vec/core/sort_description.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_map.h"
+
+// this test is gonna to be a column test template for all column which should
make ut test to coverage the function defined in column
+// for example column_array should test this function:
+// size, reserve, resize, empty, byte_size, allocated_bytes, clone_resized,
+// get_shrinked_column, filter, filter_by_selector, serialize_vec,
deserialize_vec, get_max_row_byte_size
+//
+namespace doris::vectorized {
+
+class CommonColumnTest : public ::testing::Test {
+public:
+void SetUp() override {
+col_str = ColumnString::create();
+col_str->insert_data("aaa", 3);
+col_str->insert_data("bb", 2);
+col_str->insert_data("", 4);
+
+col_int = ColumnInt64::create();
+col_int->insert_value(1);
+col_int->insert_value(2);
+col_int->insert_value(3);
+
+col_dcm = ColumnDecimal64::create(0, 3);
+col_dcm->insert_value(1.23);
+col_dcm->insert_value(4.56);
+col_dcm->insert_value(7.89);
+
+col_arr = ColumnArray::create(ColumnInt64::create(),
ColumnArray::ColumnOffsets::create());
+Array array1 = {1, 2, 3};
+Array array2 = {4};
+col_arr->insert(array1);
+col_arr->insert(Array());
+col_arr->insert(array2);
+
+col_map = ColumnMap::create(ColumnString::create(),
ColumnInt64::create(),
+ColumnArray::ColumnOffsets::create());
+Array k1 = {"a", "b", "c"};
+Array v1 = {1, 2, 3};
+Array k2 = {"d"};
+Array v2 = {4};
+Array a = Array();
+Map map1, map2, map3;
+map1.push_back(k1);
+map1.push_back(v1);
+col_map->insert(map1);
+map3.push_back(a);
+map3.push_back(a);
+col_map->insert(map3);
+map2.push_back(k2);
+map2.push_back(v2);
+col_map->insert(map2);
+}
+
+ColumnString::MutablePtr col_str;
+ColumnInt64::MutablePtr col_int;
+ColumnDecimal64::MutablePtr col_dcm;
+ColumnArray::MutablePtr col_arr;
+ColumnMap::MutablePtr col_map;
+
+void checkColumn(const IColumn& col1, const IColumn& col2, const
IDataType& dataType,
+ size_t column_size) {
+if (WhichDataType(dataType).is_map()) {
+auto map1 = check_and_get_column(col1);
+auto map2 = check_and_get_column(col2);
+const DataTypeMap& rhs_map = static_cast(dataType);
+checkColumn(map1->get_keys(), map2->get_keys(),
*rhs_map.get_key_type(),
+map1->get_keys().size());
+checkColumn(map2->get_values(), map2->get_values(),
*rhs_map.get_value_type(),
+map1->get_values().size());
+} else {
+if (WhichDataType(dataType).is_int8()) {
+auto c1 = check_and_get_column(col1);
+auto c2 = check_and_get_column(col2);
+for (size_t i = 0; i < column_size; ++i) {
+EXPECT_EQ(c1->get_element(i), c2->get_element(i));
+}
+} else if (WhichDataType(dataType).is_int16()) {
+auto c1 = check_and_get_column(col1);
+auto c2 = check_and_get_column(col2);
+for (size_t i = 0; i < column_size; ++i) {
+EXPECT_EQ(c1->get_element(i), c2->get_element(i));
+}
+} else if (WhichDataType(dataType).is_int32()) {
+auto c1 = check_and_get_column(col1);
+auto c2 =
Re: [PR] [Improve](columns) update columns and ut [doris]
amorynan commented on code in PR #42269:
URL: https://github.com/apache/doris/pull/42269#discussion_r1870515484
##
be/test/vec/columns/common_column_test.h:
##
@@ -0,0 +1,766 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include
+#include
+#include
+
+#include "olap/schema.h"
+#include "vec/columns/column.h"
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_map.h"
+#include "vec/columns/columns_number.h"
+#include "vec/core/field.h"
+#include "vec/core/sort_block.h"
+#include "vec/core/sort_description.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_map.h"
+
+// this test is gonna to be a column test template for all column which should
make ut test to coverage the function defined in column
+// for example column_array should test this function:
+// size, reserve, resize, empty, byte_size, allocated_bytes, clone_resized,
+// get_shrinked_column, filter, filter_by_selector, serialize_vec,
deserialize_vec, get_max_row_byte_size
+//
+namespace doris::vectorized {
+
+class CommonColumnTest : public ::testing::Test {
+public:
+void SetUp() override {
+col_str = ColumnString::create();
+col_str->insert_data("aaa", 3);
+col_str->insert_data("bb", 2);
+col_str->insert_data("", 4);
+
+col_int = ColumnInt64::create();
+col_int->insert_value(1);
+col_int->insert_value(2);
+col_int->insert_value(3);
+
+col_dcm = ColumnDecimal64::create(0, 3);
+col_dcm->insert_value(1.23);
+col_dcm->insert_value(4.56);
+col_dcm->insert_value(7.89);
+
+col_arr = ColumnArray::create(ColumnInt64::create(),
ColumnArray::ColumnOffsets::create());
+Array array1 = {1, 2, 3};
+Array array2 = {4};
+col_arr->insert(array1);
+col_arr->insert(Array());
+col_arr->insert(array2);
+
+col_map = ColumnMap::create(ColumnString::create(),
ColumnInt64::create(),
+ColumnArray::ColumnOffsets::create());
+Array k1 = {"a", "b", "c"};
+Array v1 = {1, 2, 3};
+Array k2 = {"d"};
+Array v2 = {4};
+Array a = Array();
+Map map1, map2, map3;
+map1.push_back(k1);
+map1.push_back(v1);
+col_map->insert(map1);
+map3.push_back(a);
+map3.push_back(a);
+col_map->insert(map3);
+map2.push_back(k2);
+map2.push_back(v2);
+col_map->insert(map2);
+}
+
+ColumnString::MutablePtr col_str;
+ColumnInt64::MutablePtr col_int;
+ColumnDecimal64::MutablePtr col_dcm;
+ColumnArray::MutablePtr col_arr;
+ColumnMap::MutablePtr col_map;
+
+void checkColumn(const IColumn& col1, const IColumn& col2, const
IDataType& dataType,
+ size_t column_size) {
+if (WhichDataType(dataType).is_map()) {
+auto map1 = check_and_get_column(col1);
+auto map2 = check_and_get_column(col2);
+const DataTypeMap& rhs_map = static_cast(dataType);
+checkColumn(map1->get_keys(), map2->get_keys(),
*rhs_map.get_key_type(),
+map1->get_keys().size());
+checkColumn(map2->get_values(), map2->get_values(),
*rhs_map.get_value_type(),
+map1->get_values().size());
+} else {
+if (WhichDataType(dataType).is_int8()) {
+auto c1 = check_and_get_column(col1);
+auto c2 = check_and_get_column(col2);
+for (size_t i = 0; i < column_size; ++i) {
+EXPECT_EQ(c1->get_element(i), c2->get_element(i));
+}
+} else if (WhichDataType(dataType).is_int16()) {
+auto c1 = check_and_get_column(col1);
+auto c2 = check_and_get_column(col2);
+for (size_t i = 0; i < column_size; ++i) {
+EXPECT_EQ(c1->get_element(i), c2->get_element(i));
+}
+} else if (WhichDataType(dataType).is_int32()) {
+auto c1 = check_and_get_column(col1);
+auto c2 =
Re: [PR] [Improve](columns) update columns and ut [doris]
amorynan commented on code in PR #42269:
URL: https://github.com/apache/doris/pull/42269#discussion_r1870515229
##
be/test/vec/columns/common_column_test.h:
##
@@ -0,0 +1,766 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include
+#include
+#include
+
+#include "olap/schema.h"
+#include "vec/columns/column.h"
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_map.h"
+#include "vec/columns/columns_number.h"
+#include "vec/core/field.h"
+#include "vec/core/sort_block.h"
+#include "vec/core/sort_description.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_map.h"
+
+// this test is gonna to be a column test template for all column which should
make ut test to coverage the function defined in column
+// for example column_array should test this function:
+// size, reserve, resize, empty, byte_size, allocated_bytes, clone_resized,
+// get_shrinked_column, filter, filter_by_selector, serialize_vec,
deserialize_vec, get_max_row_byte_size
+//
+namespace doris::vectorized {
+
+class CommonColumnTest : public ::testing::Test {
+public:
+void SetUp() override {
+col_str = ColumnString::create();
+col_str->insert_data("aaa", 3);
+col_str->insert_data("bb", 2);
+col_str->insert_data("", 4);
+
+col_int = ColumnInt64::create();
+col_int->insert_value(1);
+col_int->insert_value(2);
+col_int->insert_value(3);
+
+col_dcm = ColumnDecimal64::create(0, 3);
+col_dcm->insert_value(1.23);
+col_dcm->insert_value(4.56);
+col_dcm->insert_value(7.89);
+
+col_arr = ColumnArray::create(ColumnInt64::create(),
ColumnArray::ColumnOffsets::create());
+Array array1 = {1, 2, 3};
+Array array2 = {4};
+col_arr->insert(array1);
+col_arr->insert(Array());
+col_arr->insert(array2);
+
+col_map = ColumnMap::create(ColumnString::create(),
ColumnInt64::create(),
+ColumnArray::ColumnOffsets::create());
+Array k1 = {"a", "b", "c"};
+Array v1 = {1, 2, 3};
+Array k2 = {"d"};
+Array v2 = {4};
+Array a = Array();
+Map map1, map2, map3;
+map1.push_back(k1);
+map1.push_back(v1);
+col_map->insert(map1);
+map3.push_back(a);
+map3.push_back(a);
+col_map->insert(map3);
+map2.push_back(k2);
+map2.push_back(v2);
+col_map->insert(map2);
+}
+
+ColumnString::MutablePtr col_str;
+ColumnInt64::MutablePtr col_int;
+ColumnDecimal64::MutablePtr col_dcm;
+ColumnArray::MutablePtr col_arr;
+ColumnMap::MutablePtr col_map;
+
+void checkColumn(const IColumn& col1, const IColumn& col2, const
IDataType& dataType,
+ size_t column_size) {
+if (WhichDataType(dataType).is_map()) {
+auto map1 = check_and_get_column(col1);
+auto map2 = check_and_get_column(col2);
+const DataTypeMap& rhs_map = static_cast(dataType);
+checkColumn(map1->get_keys(), map2->get_keys(),
*rhs_map.get_key_type(),
+map1->get_keys().size());
+checkColumn(map2->get_values(), map2->get_values(),
*rhs_map.get_value_type(),
+map1->get_values().size());
+} else {
+if (WhichDataType(dataType).is_int8()) {
+auto c1 = check_and_get_column(col1);
+auto c2 = check_and_get_column(col2);
+for (size_t i = 0; i < column_size; ++i) {
+EXPECT_EQ(c1->get_element(i), c2->get_element(i));
+}
+} else if (WhichDataType(dataType).is_int16()) {
+auto c1 = check_and_get_column(col1);
+auto c2 = check_and_get_column(col2);
+for (size_t i = 0; i < column_size; ++i) {
+EXPECT_EQ(c1->get_element(i), c2->get_element(i));
+}
+} else if (WhichDataType(dataType).is_int32()) {
+auto c1 = check_and_get_column(col1);
+auto c2 =
Re: [PR] [Improve](columns) update columns and ut [doris]
github-actions[bot] commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2516107403 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
github-actions[bot] commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2516107437 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
doris-robot commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2514039967 ClickBench: Total hot run time: 32.55 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 4cb96b1d9ff9426f8827d2170fa1f54511c30cc0, data reload: false query1 0.030.040.03 query2 0.070.030.03 query3 0.230.070.07 query4 1.630.100.11 query5 0.420.380.40 query6 1.140.660.66 query7 0.010.020.01 query8 0.040.040.03 query9 0.580.510.51 query10 0.560.560.57 query11 0.140.100.10 query12 0.140.110.12 query13 0.590.610.59 query14 2.822.712.77 query15 0.900.830.83 query16 0.390.390.38 query17 1.071.061.06 query18 0.210.220.20 query19 1.991.902.03 query20 0.010.010.02 query21 15.36 0.590.58 query22 2.582.102.10 query23 16.95 1.020.75 query24 3.131.740.64 query25 0.150.060.08 query26 0.640.150.14 query27 0.050.050.05 query28 10.58 1.101.06 query29 12.57 3.313.27 query30 0.250.060.06 query31 2.870.380.37 query32 3.260.470.47 query33 2.963.043.08 query34 16.75 4.454.47 query35 4.504.444.52 query36 0.650.490.50 query37 0.090.060.06 query38 0.050.040.04 query39 0.030.020.03 query40 0.160.130.13 query41 0.080.020.02 query42 0.030.020.03 query43 0.030.020.03 Total cold run time: 106.69 s Total hot run time: 32.55 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
doris-robot commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2514015039 TPC-DS: Total hot run time: 198239 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 4cb96b1d9ff9426f8827d2170fa1f54511c30cc0, data reload: false query1 1290962 979 962 query2 6357209921022099 query3 10950 459444724472 query4 66844 29297 23495 23495 query5 5161471 448 448 query6 415 187 189 187 query7 5682314 295 295 query8 330 248 259 248 query9 9339262126152615 query10 488 242 273 242 query11 17492 15503 15825 15503 query12 153 101 101 101 query13 1600421 406 406 query14 9889735976327359 query15 217 184 201 184 query16 7711505 507 505 query17 1103624 618 618 query18 1864308 311 308 query19 229 161 159 159 query20 120 115 117 115 query21 228 119 105 105 query22 4684465946314631 query23 35026 34660 34427 34427 query24 5531256624272427 query25 510 417 428 417 query26 643 158 145 145 query27 1724292 299 292 query28 4147245724522452 query29 690 422 422 422 query30 209 150 146 146 query31 980 839 846 839 query32 70 55 57 55 query33 433 313 290 290 query34 938 507 528 507 query35 872 784 803 784 query36 1103977 973 973 query37 120 79 72 72 query38 4452447845234478 query39 1547146414531453 query40 194 96 99 96 query41 45 43 47 43 query42 114 98 103 98 query43 565 508 514 508 query44 1229822 843 822 query45 190 169 172 169 query46 1174709 737 709 query47 2037193919251925 query48 434 323 336 323 query49 725 396 396 396 query50 855 391 400 391 query51 7378719971337133 query52 101 86 95 86 query53 255 178 182 178 query54 523 410 398 398 query55 80 83 81 81 query56 248 246 228 228 query57 1268109311271093 query58 220 202 215 202 query59 3352301830973018 query60 268 242 257 242 query61 113 113 110 110 query62 811 660 660 660 query63 221 191 210 191 query64 1350677 649 649 query65 3365318432013184 query66 721 294 299 294 query67 15869 15606 15714 15606 query68 4456572 571 571 query69 407 261 254 254 query70 1138115910921092 query71 377 253 249 249 query72 6519404540914045 query73 774 355 361 355 query74 10324 902090319020 query75 3390267626892676 query76 2101114010791079 query77 459 275 266 266 query78 10362 952895239523 query79 1334636 611 611 query80 883 442 437 437 query81 484 246 228 228 query82 1147125 118 118 query83 161 144 152 144 query84 271 72 66 66 query85 862 306 301 301 query86 333 314 303 303 query87 4801480645274527 query88 3330218321362136 query89 412 301 292 292 query90 2048189 186 186 query91 138 102 105 102 query92 61 52 51 51 query93 1547553 559 553 query94 783 272 294 272 query95 345 254 246 246 query96 624 273 278 273 query97 2830269926962696 query98 220 197 193 193 query99 1556132813051305 Total cold run time: 319323 ms Total hot run time: 198239 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [Improve](columns) update columns and ut [doris]
doris-robot commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2513986116 TPC-H: Total hot run time: 40254 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 4cb96b1d9ff9426f8827d2170fa1f54511c30cc0, data reload: false -- Round 1 -- q1 17643 757373737373 q2 2056178 179 178 q3 10658 108711991087 q4 10580 706 809 706 q5 7629276927032703 q6 239 147 150 147 q7 1016658 588 588 q8 9243194119221922 q9 6654645065036450 q10 7002234823762348 q11 457 261 263 261 q12 428 228 221 221 q13 17795 307230243024 q14 250 210 227 210 q15 564 529 513 513 q16 617 577 595 577 q17 993 625 545 545 q18 7285681968076807 q19 14001011933 933 q20 462 181 179 179 q21 4058331231643164 q22 390 318 319 318 Total cold run time: 107419 ms Total hot run time: 40254 ms - Round 2, with runtime_filter_mode=off - q1 7228729072637263 q2 332 235 230 230 q3 2939286629832866 q4 2043182018471820 q5 5683568856845684 q6 219 139 140 139 q7 2248183218531832 q8 3418355235843552 q9 8915898990398989 q10 3615357635683568 q11 595 497 511 497 q12 823 605 625 605 q13 10996 324132033203 q14 313 276 274 274 q15 573 539 517 517 q16 688 637 667 637 q17 1888168416331633 q18 8419768676757675 q19 1707163715711571 q20 2131190519161905 q21 5672542154605421 q22 642 625 610 610 Total cold run time: 71087 ms Total hot run time: 60491 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
amorynan commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2513808985 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
amorynan commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2513761781 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
doris-robot commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2511762694 ClickBench: Total hot run time: 33.2 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 8f9556589e2da008168b7b54c73666937db20b74, data reload: false query1 0.030.030.03 query2 0.060.030.03 query3 0.240.080.07 query4 1.620.100.10 query5 0.440.410.43 query6 1.170.660.66 query7 0.020.010.01 query8 0.050.030.03 query9 0.570.510.51 query10 0.560.560.56 query11 0.140.100.11 query12 0.130.120.11 query13 0.610.600.62 query14 2.692.732.83 query15 0.910.840.83 query16 0.380.390.37 query17 1.001.071.03 query18 0.220.220.22 query19 1.871.892.01 query20 0.010.010.01 query21 15.37 0.600.60 query22 2.582.561.75 query23 17.06 0.880.84 query24 2.611.131.48 query25 0.360.210.12 query26 0.370.140.14 query27 0.050.050.05 query28 10.75 1.081.07 query29 12.55 3.523.53 query30 0.250.070.06 query31 2.870.390.38 query32 3.250.460.48 query33 2.973.063.06 query34 16.97 4.464.46 query35 4.464.474.52 query36 0.670.490.48 query37 0.090.060.06 query38 0.040.040.03 query39 0.030.020.02 query40 0.170.140.13 query41 0.080.030.02 query42 0.040.020.03 query43 0.030.030.03 Total cold run time: 106.34 s Total hot run time: 33.2 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
doris-robot commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2511741124 TeamCity be ut coverage result: Function Coverage: 38.44% (9988/25985) Line Coverage: 29.49% (83781/284056) Region Coverage: 28.61% (43096/150608) Branch Coverage: 25.24% (21917/86830) Coverage Report: http://coverage.selectdb-in.cc/coverage/8f9556589e2da008168b7b54c73666937db20b74_8f9556589e2da008168b7b54c73666937db20b74/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
doris-robot commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2511748427 TPC-DS: Total hot run time: 190633 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 8f9556589e2da008168b7b54c73666937db20b74, data reload: false query1 972 394 378 378 query2 6520203520392035 query3 6714217 209 209 query4 33768 23615 23667 23615 query5 4322463 446 446 query6 271 184 173 173 query7 4618314 310 310 query8 301 253 237 237 query9 9518264226552642 query10 458 240 264 240 query11 18173 15539 15245 15245 query12 167 106 109 106 query13 1676419 396 396 query14 11443 662370386623 query15 321 185 177 177 query16 7944457 474 457 query17 1783582 585 582 query18 2125306 304 304 query19 374 161 156 156 query20 121 115 112 112 query21 210 106 105 105 query22 4470422342494223 query23 35013 33942 34179 33942 query24 11373 250224422442 query25 651 375 380 375 query26 1816149 152 149 query27 2866284 281 281 query28 7975242224212421 query29 1008402 392 392 query30 298 150 148 148 query31 1061825 823 823 query32 93 54 57 54 query33 766 283 283 283 query34 1007509 535 509 query35 866 733 758 733 query36 1105969 954 954 query37 277 73 77 73 query38 4408438642144214 query39 1481141114251411 query40 281 101 100 100 query41 45 49 45 45 query42 107 97 99 97 query43 542 488 486 486 query44 1251822 815 815 query45 190 165 165 165 query46 1187738 712 712 query47 1987188918621862 query48 421 309 314 309 query49 1298409 411 409 query50 803 395 386 386 query51 7289709870967096 query52 104 91 85 85 query53 250 180 178 178 query54 1205397 414 397 query55 79 80 81 80 query56 256 235 239 235 query57 1228113811181118 query58 240 213 222 213 query59 3347296628322832 query60 267 261 239 239 query61 110 106 106 106 query62 860 699 672 672 query63 222 181 180 180 query64 5035660 634 634 query65 3331319532203195 query66 1420331 327 327 query67 16238 15602 15706 15602 query68 5303543 568 543 query69 401 244 252 244 query70 1211103111301031 query71 335 247 249 247 query72 6341412540214021 query73 777 360 357 357 query74 10453 897389768973 query75 3458264726552647 query76 3184113011571130 query77 516 262 297 262 query78 10595 961394739473 query79 1175590 588 588 query80 1148434 425 425 query81 527 231 229 229 query82 1103115 114 114 query83 233 160 144 144 query84 240 72 69 69 query85 1252293 296 293 query86 378 284 303 284 query87 4789461645864586 query88 3404224921882188 query89 404 296 295 295 query90 2086191 192 191 query91 138 102 104 102 query92 59 54 50 50 query93 1083541 543 541 query94 1147291 296 291 query95 362 244 258 244 query96 617 276 282 276 query97 2890272427682724 query98 222 211 192 192 query99 1564132913451329 Total cold run time: 305321 ms Total hot run time: 190633 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [Improve](columns) update columns and ut [doris]
amorynan commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2511669183 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
doris-robot commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2511717837 TPC-H: Total hot run time: 40371 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 8f9556589e2da008168b7b54c73666937db20b74, data reload: false -- Round 1 -- q1 17709 820172717271 q2 2047175 163 163 q3 10629 113612041136 q4 10230 726 794 726 q5 7609277227552755 q6 239 148 147 147 q7 1022623 611 611 q8 9242193819101910 q9 6714653565476535 q10 6992228622942286 q11 463 269 262 262 q12 435 229 231 229 q13 17769 305730123012 q14 253 211 219 211 q15 575 539 514 514 q16 648 595 596 595 q17 1001603 511 511 q18 7467673968626739 q19 1328985 989 985 q20 473 177 173 173 q21 4051328633163286 q22 391 314 328 314 Total cold run time: 107287 ms Total hot run time: 40371 ms - Round 2, with runtime_filter_mode=off - q1 7253728273127282 q2 327 236 230 230 q3 2962276127772761 q4 2008171217381712 q5 5433541954415419 q6 225 134 136 134 q7 2146171817061706 q8 3241339234193392 q9 8716862786468627 q10 3496345334513451 q11 597 495 491 491 q12 807 604 578 578 q13 10546 306030453045 q14 294 259 253 253 q15 565 516 512 512 q16 682 630 618 618 q17 1798158315921583 q18 7850732575227325 q19 1679152615761526 q20 2087184218161816 q21 5379533153745331 q22 632 572 573 572 Total cold run time: 68723 ms Total hot run time: 58364 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
doris-robot commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2511146083 ClickBench: Total hot run time: 32.81 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 99d08790312a3e79417791aff516429033a5809e, data reload: false query1 0.040.030.03 query2 0.070.030.03 query3 0.230.080.08 query4 1.610.100.11 query5 0.410.430.42 query6 1.170.660.64 query7 0.020.020.01 query8 0.030.030.03 query9 0.580.510.49 query10 0.560.560.57 query11 0.130.100.10 query12 0.140.110.12 query13 0.600.600.60 query14 2.762.762.76 query15 0.900.820.81 query16 0.400.390.39 query17 1.031.000.98 query18 0.230.210.21 query19 1.961.841.87 query20 0.010.020.01 query21 15.36 0.600.58 query22 2.641.771.59 query23 17.03 0.860.86 query24 3.521.461.41 query25 0.260.190.04 query26 0.610.130.14 query27 0.050.040.05 query28 9.891.111.07 query29 12.58 3.283.25 query30 0.240.060.06 query31 2.860.390.38 query32 3.260.460.48 query33 3.023.023.17 query34 17.13 4.454.50 query35 4.554.514.51 query36 0.680.500.46 query37 0.090.050.06 query38 0.040.030.04 query39 0.030.020.02 query40 0.170.140.13 query41 0.080.030.02 query42 0.040.020.02 query43 0.040.040.03 Total cold run time: 107.05 s Total hot run time: 32.81 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
doris-robot commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2511121144 TeamCity be ut coverage result: Function Coverage: 38.43% (9986/25985) Line Coverage: 29.49% (83763/284056) Region Coverage: 28.61% (43085/150608) Branch Coverage: 25.24% (21913/86830) Coverage Report: http://coverage.selectdb-in.cc/coverage/99d08790312a3e79417791aff516429033a5809e_99d08790312a3e79417791aff516429033a5809e/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
doris-robot commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2511132411 TPC-DS: Total hot run time: 196906 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 99d08790312a3e79417791aff516429033a5809e, data reload: false query1 1274942 937 937 query2 6237202720342027 query3 10924 444344304430 query4 66803 27882 23664 23664 query5 4969463 444 444 query6 395 189 180 180 query7 5508308 306 306 query8 327 235 241 235 query9 8292263926452639 query10 405 236 233 233 query11 17140 15122 15993 15122 query12 153 104 100 100 query13 1438416 395 395 query14 9715739571627162 query15 207 182 181 181 query16 7211472 499 472 query17 1040590 559 559 query18 1828293 290 290 query19 223 165 163 163 query20 122 112 107 107 query21 203 102 108 102 query22 4792456345914563 query23 34907 34257 34423 34257 query24 5392247924802479 query25 505 415 429 415 query26 658 154 152 152 query27 1858278 283 278 query28 4706248824962488 query29 705 445 459 445 query30 215 155 155 155 query31 1009843 824 824 query32 73 56 60 56 query33 445 315 302 302 query34 939 514 515 514 query35 910 772 824 772 query36 1114956 973 956 query37 128 75 77 75 query38 4459438344574383 query39 1521143915321439 query40 207 96 96 96 query41 43 42 42 42 query42 109 96 100 96 query43 516 507 494 494 query44 1195830 839 830 query45 187 166 170 166 query46 1184726 753 726 query47 2054192819151915 query48 420 345 323 323 query49 711 392 398 392 query50 847 409 400 400 query51 7462708169686968 query52 99 87 87 87 query53 259 175 174 174 query54 506 424 394 394 query55 72 73 71 71 query56 253 229 240 229 query57 1240114010861086 query58 207 204 215 204 query59 3076304328922892 query60 258 243 238 238 query61 103 108 109 108 query62 773 680 664 664 query63 216 181 182 181 query64 1358676 639 639 query65 3281318532303185 query66 713 301 294 294 query67 16002 15848 15636 15636 query68 3952572 571 571 query69 423 247 254 247 query70 1183114411391139 query71 361 248 250 248 query72 6407413340504050 query73 748 361 365 361 query74 10235 903890279027 query75 3403266126462646 query76 1810103611101036 query77 452 272 276 272 query78 10525 946894639463 query79 1531600 579 579 query80 878 424 429 424 query81 509 247 222 222 query82 1273120 117 117 query83 246 145 145 145 query84 271 66 64 64 query85 886 329 296 296 query86 343 303 297 297 query87 4779451146494511 query88 3763219321802180 query89 411 287 291 287 query90 2035186 188 186 query91 130 101 102 101 query92 73 52 49 49 query93 2005543 540 540 query94 808 292 279 279 query95 357 249 250 249 query96 629 272 281 272 query97 2873273527132713 query98 221 204 193 193 query99 1664133413111311 Total cold run time: 317285 ms Total hot run time: 196906 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [Improve](columns) update columns and ut [doris]
doris-robot commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2511102662 TPC-H: Total hot run time: 40212 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 99d08790312a3e79417791aff516429033a5809e, data reload: false -- Round 1 -- q1 17586 752673287328 q2 2051174 164 164 q3 10728 107911411079 q4 10253 768 777 768 q5 7632273126782678 q6 240 147 147 147 q7 1028628 594 594 q8 9238191119431911 q9 6682652664906490 q10 6987230522942294 q11 464 265 261 261 q12 429 224 219 219 q13 17760 306030253025 q14 249 208 235 208 q15 579 535 533 533 q16 655 564 581 564 q17 1001548 609 548 q18 7222682266696669 q19 1358102510111011 q20 456 182 175 175 q21 4028324832293229 q22 372 317 320 317 Total cold run time: 106998 ms Total hot run time: 40212 ms - Round 2, with runtime_filter_mode=off - q1 7293724972797249 q2 329 223 232 223 q3 2947279329652793 q4 2062185618241824 q5 5688568756775677 q6 222 145 144 144 q7 2213183318201820 q8 3395359435263526 q9 8916919690399039 q10 3599359535703570 q11 599 514 521 514 q12 830 610 592 592 q13 12636 328331903190 q14 315 267 284 267 q15 577 523 532 523 q16 684 626 665 626 q17 1866165216511651 q18 8425772876887688 q19 1715160315851585 q20 2109187718611861 q21 5689547855995478 q22 648 587 576 576 Total cold run time: 72757 ms Total hot run time: 60416 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
amorynan commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2510886329 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
doris-robot commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2507778735 ClickBench: Total hot run time: 31.91 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 249f359b18a233eea91b7fc49f20431bbac7fcf0, data reload: false query1 0.030.030.04 query2 0.060.030.03 query3 0.240.070.07 query4 1.620.100.11 query5 0.410.410.42 query6 1.130.670.66 query7 0.020.010.02 query8 0.040.030.03 query9 0.580.500.50 query10 0.550.560.55 query11 0.150.110.10 query12 0.140.120.11 query13 0.610.600.60 query14 2.842.752.85 query15 0.910.820.83 query16 0.390.380.38 query17 1.060.991.00 query18 0.220.200.21 query19 1.981.861.96 query20 0.010.010.01 query21 15.36 0.580.58 query22 3.002.221.41 query23 17.11 1.030.78 query24 3.240.751.39 query25 0.160.110.17 query26 0.490.140.14 query27 0.040.040.04 query28 10.64 1.091.08 query29 12.61 3.223.19 query30 0.260.080.06 query31 3.280.390.39 query32 3.460.480.49 query33 2.993.023.00 query34 16.89 4.454.52 query35 4.584.444.43 query36 0.670.510.49 query37 0.080.060.06 query38 0.040.040.04 query39 0.030.020.02 query40 0.150.130.13 query41 0.080.020.02 query42 0.030.020.02 query43 0.030.030.03 Total cold run time: 108.21 s Total hot run time: 31.91 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
doris-robot commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2507769870 TPC-DS: Total hot run time: 197847 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 249f359b18a233eea91b7fc49f20431bbac7fcf0, data reload: false query1 1245967 936 936 query2 6244213621282128 query3 11017 436644124366 query4 3 28644 23750 23750 query5 4935471 488 471 query6 417 184 176 176 query7 5544310 305 305 query8 322 240 243 240 query9 8499268326182618 query10 438 252 247 247 query11 17214 15177 15944 15177 query12 153 99 103 99 query13 1452441 410 410 query14 10552 729576837295 query15 233 190 193 190 query16 7086420 473 420 query17 1095574 577 574 query18 1717306 304 304 query19 195 156 160 156 query20 117 112 110 110 query21 204 103 104 103 query22 4878450444094409 query23 34707 3 34614 34614 query24 5466253126182531 query25 479 397 409 397 query26 630 161 155 155 query27 1778295 297 295 query28 4743250924802480 query29 655 420 421 420 query30 207 158 153 153 query31 1026849 876 849 query32 76 57 59 57 query33 461 327 315 315 query34 951 530 523 523 query35 881 776 775 775 query36 1064980 993 980 query37 135 79 81 79 query38 4638448444554455 query39 1527145216181452 query40 209 110 100 100 query41 46 42 44 42 query42 118 101 97 97 query43 568 534 521 521 query44 1249825 832 825 query45 190 179 173 173 query46 1161767 743 743 query47 2007192519581925 query48 454 311 332 311 query49 729 405 411 405 query50 870 397 410 397 query51 7364715969676967 query52 95 87 86 86 query53 261 182 186 182 query54 520 398 401 398 query55 80 80 78 78 query56 253 235 234 234 query57 1228112111331121 query58 242 222 216 216 query59 3290299628942894 query60 274 251 247 247 query61 106 104 105 104 query62 787 665 674 665 query63 218 189 200 189 query64 1373737 649 649 query65 3270322432413224 query66 703 303 307 303 query67 16034 15716 15436 15436 query68 4247573 581 573 query69 434 251 257 251 query70 1202117211311131 query71 356 255 252 252 query72 6100418640914091 query73 772 361 358 358 query74 10318 902890389028 query75 3392268526912685 query76 1884117711871177 query77 574 288 308 288 query78 10434 937994149379 query79 2888617 623 617 query80 1364431 446 431 query81 521 232 233 232 query82 766 129 119 119 query83 172 151 143 143 query84 283 77 68 68 query85 1021303 300 300 query86 406 313 302 302 query87 4771458445984584 query88 4234220821392139 query89 411 305 294 294 query90 2045193 188 188 query91 141 103 108 103 query92 67 51 52 51 query93 2638553 559 553 query94 909 284 299 284 query95 355 250 254 250 query96 628 282 274 274 query97 2859266027272660 query98 216 203 198 198 query99 1584130112981298 Total cold run time: 321335 ms Total hot run time: 197847 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [Improve](columns) update columns and ut [doris]
doris-robot commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2507747270 TPC-H: Total hot run time: 40080 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 249f359b18a233eea91b7fc49f20431bbac7fcf0, data reload: false -- Round 1 -- q1 17731 750973187318 q2 2049173 163 163 q3 10993 109312531093 q4 10346 763 749 749 q5 7616269126292629 q6 240 151 150 150 q7 1024646 613 613 q8 9237190020141900 q9 6682652265156515 q10 7003231523082308 q11 452 260 265 260 q12 445 233 230 230 q13 17805 310430573057 q14 260 214 226 214 q15 571 522 512 512 q16 648 580 583 580 q17 1001602 564 564 q18 7530675568726755 q19 1351981 992 981 q20 479 186 181 181 q21 4039318029892989 q22 378 319 321 319 Total cold run time: 107880 ms Total hot run time: 40080 ms - Round 2, with runtime_filter_mode=off - q1 7221726272917262 q2 328 233 235 233 q3 2931287429612874 q4 2114181418611814 q5 5979573857595738 q6 230 144 148 144 q7 2282183318151815 q8 3440358036473580 q9 8962919491299129 q10 3618357835433543 q11 614 506 503 503 q12 822 643 642 642 q13 10859 326932723269 q14 297 290 290 290 q15 571 526 529 526 q16 699 636 642 636 q17 1882161516131613 q18 8334784576567656 q19 1717166614911491 q20 2119187318581858 q21 5726549355375493 q22 651 548 561 548 Total cold run time: 71396 ms Total hot run time: 60657 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
amorynan commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2507563987 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
amorynan commented on code in PR #42269:
URL: https://github.com/apache/doris/pull/42269#discussion_r1859651107
##
be/src/vec/columns/column.h:
##
@@ -481,11 +481,12 @@ class IColumn : public COW {
virtual Ptr replicate(const Offsets& offsets) const = 0;
/// Appends one field multiple times. Can be optimized in inherited
classes.
-virtual void insert_many(const Field& field, size_t length) {
-for (size_t i = 0; i < length; ++i) {
-insert(field);
-}
-}
+// this function has not used ??
Review Comment:
just done!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
eldenmoon commented on code in PR #42269:
URL: https://github.com/apache/doris/pull/42269#discussion_r1857996603
##
be/src/vec/columns/column.h:
##
@@ -481,11 +481,12 @@ class IColumn : public COW {
virtual Ptr replicate(const Offsets& offsets) const = 0;
/// Appends one field multiple times. Can be optimized in inherited
classes.
-virtual void insert_many(const Field& field, size_t length) {
-for (size_t i = 0; i < length; ++i) {
-insert(field);
-}
-}
+// this function has not used ??
Review Comment:
if not used remove
##
be/test/vec/columns/column_ip_test.cpp:
##
@@ -0,0 +1,330 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
Review Comment:
column_ip_test.cpp lgtm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
HappenLee commented on code in PR #42269:
URL: https://github.com/apache/doris/pull/42269#discussion_r1858009105
##
be/src/vec/columns/column.h:
##
@@ -481,11 +481,12 @@ class IColumn : public COW {
virtual Ptr replicate(const Offsets& offsets) const = 0;
/// Appends one field multiple times. Can be optimized in inherited
classes.
-virtual void insert_many(const Field& field, size_t length) {
-for (size_t i = 0; i < length; ++i) {
-insert(field);
-}
-}
+// this function has not used ??
Review Comment:
just delete the code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
github-actions[bot] commented on code in PR #42269:
URL: https://github.com/apache/doris/pull/42269#discussion_r1840391583
##
be/test/vec/columns/common_column_test.h:
##
@@ -0,0 +1,1682 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include
+#include
+#include
+
+#include
+
+#include "olap/schema.h"
+#include "vec/columns/column.h"
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_map.h"
+#include "vec/columns/columns_number.h"
+#include "vec/core/field.h"
+#include "vec/core/sort_block.h"
+#include "vec/core/sort_description.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_map.h"
+
+// this test is gonna to be a column test template for all column which should
make ut test to coverage the function defined in column
+// for example column_array should test this function:
+// size, reserve, resize, empty, byte_size, allocated_bytes, clone_resized,
+// get_shrinked_column, filter, filter_by_selector, serialize_vec,
deserialize_vec, get_max_row_byte_size
+//
+namespace doris::vectorized {
+
+static bool gen_check_data_in_assert = false;
+
+class CommonColumnTest : public ::testing::Test {
+protected:
+// Helper function to load data from CSV, with index which splited by
spliter and load to columns
+void load_data_from_csv(const DataTypeSerDeSPtrs serders, MutableColumns&
columns,
+const std::string& file_path, const char spliter =
';',
+const std::set idxes = {0}) {
+ASSERT_EQ(serders.size(), columns.size())
+<< "serder size: " << serders.size() << " column size: " <<
columns.size();
+ASSERT_EQ(serders.size(), idxes.size())
+<< "serder size: " << serders.size() << " idxes size: " <<
idxes.size();
+ASSERT_EQ(serders.size(), *idxes.end())
+<< "serder size: " << serders.size() << " idxes size: " <<
*idxes.end();
+std::ifstream file(file_path);
+if (!file) {
+throw doris::Exception(ErrorCode::INVALID_ARGUMENT, "can not open
the file: {} ",
+ file_path);
+}
+
+std::string line;
+DataTypeSerDe::FormatOptions options;
+while (std::getline(file, line)) {
+std::stringstream lineStream(line);
+std::cout << "whole : " << lineStream.str() << std::endl;
+std::string value;
+int l_idx = 0;
+int c_idx = 0;
+while (std::getline(lineStream, value, spliter)) {
+if (idxes.contains(l_idx)) {
+Slice string_slice(value.data(), value.size());
+std::cout << string_slice << std::endl;
+if (auto st =
serders[c_idx]->deserialize_one_cell_from_json(
+*columns[c_idx], string_slice, options);
+!st.ok()) {
+std::cout << "error in deserialize but continue: " <<
st.to_string()
+ << std::endl;
+}
+++c_idx;
+}
+++l_idx;
+}
+}
+}
+
+ this is very helpful function to check data in column against
expected results according different function in assert function
+ such as run regress tests
+ if gen_check_data_in_assert is true, we will generate a file for
check data, otherwise we will read the file to check data
+ so the key point is we should how we write assert callback function
to check data,
+/// and when check data is generated, we should check result to statisfy
the semantic of the function
+static void check_res_file(string function_name,
std::vector>& res) {
+string filename = "./res_" + function_name + ".csv";
+if (gen_check_data_in_assert) {
+std::ofstream res_file(filename);
+std::cout << "gen check data: " << res.size() << " with file: " <<
filename
+ << std::endl;
+if (!res_file.is_open()) {
+throw std::ios
Re: [PR] [Improve](columns) update columns and ut [doris]
github-actions[bot] commented on code in PR #42269:
URL: https://github.com/apache/doris/pull/42269#discussion_r1839768565
##
be/test/vec/columns/common_column_test.h:
##
@@ -0,0 +1,1650 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include
+#include
+#include
+#include
+
+#include "olap/schema.h"
+#include "vec/columns/column.h"
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_map.h"
+#include "vec/columns/columns_number.h"
+#include "vec/core/field.h"
+#include "vec/core/sort_block.h"
+#include "vec/core/sort_description.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_map.h"
+
+// this test is gonna to be a column test template for all column which should
make ut test to coverage the function defined in column
+// for example column_array should test this function:
+// size, reserve, resize, empty, byte_size, allocated_bytes, clone_resized,
+// get_shrinked_column, filter, filter_by_selector, serialize_vec,
deserialize_vec, get_max_row_byte_size
+//
+namespace doris::vectorized {
+
+class CommonColumnTest : public ::testing::Test {
+protected:
+// Helper function to load data from CSV, with index which splited by
spliter and return a vector of ColumnPtr
+void load_data_from_csv(const DataTypeSerDeSPtrs serders, MutableColumns&
columns,
+const std::string& file_path, const char spliter =
';', const std::set idxes = {0}, bool gen_check_data =false ) {
+
+ASSERT_EQ(serders.size(), columns.size()) << "serder size: " <<
serders.size() << " column size: " << columns.size();
+ASSERT_EQ(serders.size(), idxes.size()) << "serder size: " <<
serders.size() << " idxes size: " << idxes.size();
+ASSERT_EQ(serders.size(), *idxes.end()) << "serder size: " <<
serders.size() << " idxes size: " << *idxes.end();
+std::ifstream file(file_path);
+if (!file) {
+throw doris::Exception(ErrorCode::INVALID_ARGUMENT, "can not open
the file: {} ", file_path);
+}
+
+std::string line;
+DataTypeSerDe::FormatOptions options;
+std::vector> res;
+while (std::getline(file, line)) {
+std::stringstream lineStream(line);
+
+std::cout << "whole : " << lineStream.str() << std::endl;
+std::string value;
+int l_idx = 0;
+int c_idx = 0;
+std::vector data;
+while (std::getline(lineStream, value, spliter)) {
+if (idxes.contains(l_idx)) {
+Slice string_slice(value.data(), value.size());
+std::cout << string_slice << std::endl;
+if (auto st =
serders[c_idx]->deserialize_one_cell_from_json(*columns[c_idx], string_slice,
options); !st.ok()) {
+data.push_back("");
+std::cout << "error in deserialize but continue: " <<
st.to_string() << std::endl;
+} else {
+data.push_back(value);
+}
+++ c_idx;
+}
+++ l_idx;
+}
+res.push_back(data);
+}
+if (gen_check_data) {
+string filename = "./res.csv";
+std::ofstream res_file(filename);
+std::cout << "gen check data: " << res.size() <>& res, const
std::string& file_path, const char spliter = ';') {
+std::ifstream file(file_path);
+if (!file) {
+throw doris::Exception(ErrorCode::INVALID_ARGUMENT, "can not open
the file: {} ", file_path);
+}
+std::string line;
+
+while (std::getline(file, line)) {
+std::vector data;
+std::stringstream ss(line);
+std::string value;
+while (std::getline(ss, value, spliter)) {
+data.push_back(value);
+}
+res.push_back(data);
+}
+}
+
+private:
+
+public:
+void SetUp() override {
+col_str = ColumnString::create();
+col_str->insert_data("aaa", 3);
+col_str->insert_data("bb", 2)
Re: [PR] [Improve](columns) update columns and ut [doris]
github-actions[bot] commented on code in PR #42269:
URL: https://github.com/apache/doris/pull/42269#discussion_r1838091494
##
be/test/vec/columns/column_ip_test.cpp:
##
@@ -0,0 +1,263 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include
Review Comment:
warning: 'gtest/gtest-message.h' file not found [clang-diagnostic-error]
```cpp
#include
^
```
##
be/test/vec/columns/common_column_test.h:
##
@@ -0,0 +1,1717 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include
+#include
+#include
+
+#include
+
+#include "olap/schema.h"
+#include "vec/columns/column.h"
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_map.h"
+#include "vec/columns/columns_number.h"
+#include "vec/core/field.h"
+#include "vec/core/sort_block.h"
+#include "vec/core/sort_description.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_map.h"
+
+// this test is gonna to be a column test template for all column which should
make ut test to coverage the function defined in column
+// for example column_array should test this function:
+// size, reserve, resize, empty, byte_size, allocated_bytes, clone_resized,
+// get_shrinked_column, filter, filter_by_selector, serialize_vec,
deserialize_vec, get_max_row_byte_size
+//
+namespace doris::vectorized {
+
+class CommonColumnTest : public ::testing::Test {
+protected:
+// Helper function to load data from CSV, with index which splited by
spliter and return a vector of ColumnPtr
+void load_data_from_csv(const DataTypeSerDeSPtrs serders, MutableColumns&
columns,
+const std::string& file_path, const char spliter =
';',
+const std::set idxes = {0}, bool
gen_check_data = false) {
+ASSERT_EQ(serders.size(), columns.size())
+<< "serder size: " << serders.size() << " column size: " <<
columns.size();
+ASSERT_EQ(serders.size(), idxes.size())
+<< "serder size: " << serders.size() << " idxes size: " <<
idxes.size();
+ASSERT_EQ(serders.size(), *idxes.end())
+<< "serder size: " << serders.size() << " idxes size: " <<
*idxes.end();
+std::ifstream file(file_path);
+if (!file) {
+throw doris::Exception(ErrorCode::INVALID_ARGUMENT, "can not open
the file: {} ",
+ file_path);
+}
+
+std::string line;
+DataTypeSerDe::FormatOptions options;
+std::vector> res;
+while (std::getline(file, line)) {
+std::stringstream lineStream(line);
+
+std::cout << "whole : " << lineStream.str() << std::endl;
+std::string value;
+int l_idx = 0;
+int c_idx = 0;
+std::vector data;
+while (std::getline(lineStream, value, spliter)) {
+if (idxes.contains(l_idx)) {
+Slice string_slice(value.data(), value.size());
+std::cout << string_slice << std::endl;
+if (auto st =
serders[c_idx]->deserialize_one_cell_from_json(
+*columns[c_idx], string_slice, options);
+!st.ok()) {
+data.push_back("");
+std::cout << "error in deserialize bu
Re: [PR] [Improve](columns) update columns and ut [doris]
eldenmoon commented on code in PR #42269:
URL: https://github.com/apache/doris/pull/42269#discussion_r1830282961
##
be/src/vec/columns/predicate_column.h:
##
@@ -157,6 +157,8 @@ class PredicateColumnType final : public COWHelperhttp://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "vec/columns/column_array.h"
+
+#include
+#include
+#include
+
+#include "vec/columns/column.h"
+#include "vec/columns/columns_number.h"
+#include "vec/columns/common_column_test.h"
+#include "vec/core/field.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+
+// this test is gonna to be a column test template for all column which should
make ut test to coverage the function defined in column
+// for example column_array should test this function:
+// size, reserve, resize, empty, byte_size, allocated_bytes, clone_resized,
+// get_shrinked_column, filter, filter_by_selector, serialize_vec,
deserialize_vec, get_max_row_byte_size
+//
+namespace doris::vectorized {
+class ColumnArrayTest : public CommonColumnTest {
+protected:
+void dump_size(ColumnArray::MutablePtr& arr) {
+std::cout << "size: " << arr->size() << std::endl;
+std::cout << "offset size: " << arr->get_offsets().size() << std::endl;
+std::cout << "data size: " << arr->get_data_ptr()->size() << std::endl;
+std::cout << "allocated_bytes: " << arr->allocated_bytes() <<
std::endl;
+std::cout << "byte_size: " << arr->byte_size() << std::endl;
+}
+
+// in array we should always check size with data and offset column
+void check_size(IColumn& arr, size_t expect_size) {
+auto& a = assert_cast(arr);
+EXPECT_EQ(a.size(), expect_size);
+EXPECT_EQ(a.get_offsets().size(), expect_size);
+EXPECT_EQ(a.get_offsets().back(), a.get_data_ptr()->size());
+}
+
+void SetUp() override {
+col_int_arr =
+ColumnArray::create(ColumnInt64::create(),
ColumnArray::ColumnOffsets::create());
+Array array1 = {1, 2, 3};
Review Comment:
add null value in array like [1, 2, null, 3], [null, null, null] etc
##
be/test/vec/columns/column_array_test.cpp:
##
@@ -0,0 +1,174 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "vec/columns/column_array.h"
+
+#include
+#include
+#include
+
+#include "vec/columns/column.h"
+#include "vec/columns/columns_number.h"
+#include "vec/columns/common_column_test.h"
+#include "vec/core/field.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+
+// this test is gonna to be a column test template for all column which should
make ut test to coverage the function defined in column
+// for example column_array should test this function:
+// size, reserve, resize, empty, byte_size, allocated_bytes, clone_resized,
+// get_shrinked_column, filter, filter_by_selector, serialize_vec,
deserialize_vec, get_max_row_byte_size
+//
+namespace doris::vectorized {
+class ColumnArrayTest : public CommonColumnTest {
+protected:
+void dump_size(ColumnArray::MutablePtr& arr) {
+std::cout << "size: " << arr->size() << std::endl;
+std::cout << "offset size: " << arr->get_offsets().size() << std::endl;
+std::cout << "data size: " << arr->get_data_ptr()->size() << std::endl;
+std::cout << "allocated_bytes: " << arr->allocated_bytes() <<
std::endl;
+std::cout << "byte_size: " << arr->byte_size() << std::endl;
+}
+
+// in array we should always check size with data and offset column
+void check_size(IColumn& arr, size_t expect_size) {
+auto& a = assert_cast(arr);
+EXPECT_EQ(a.size(), expect_size);
+EXPECT_EQ(a.get_offsets().size(), expect_size);
+EXPECT_EQ(a.get_offsets().back(), a.get_data_ptr()->size());
+}
+
+void SetUp() override {
+col_int_arr =
Review Comment:
add array for
Re: [PR] [Improve](columns) update columns and ut [doris]
eldenmoon commented on code in PR #42269:
URL: https://github.com/apache/doris/pull/42269#discussion_r1830280977
##
be/src/vec/columns/column_array.cpp:
##
@@ -417,7 +418,8 @@ void ColumnArray::pop_back(size_t n) {
void ColumnArray::reserve(size_t n) {
get_offsets().reserve(n);
get_data().reserve(
-n); /// The average size of arrays is not taken into account here.
Or it is considered to be no more than 1.
+get_offsets()
Review Comment:
why modify this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [Improve](columns) update columns and ut [doris]
github-actions[bot] commented on code in PR #42269:
URL: https://github.com/apache/doris/pull/42269#discussion_r1817969502
##
be/test/vec/columns/common_column_test.h:
##
@@ -0,0 +1,766 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include
+#include
+#include
+
+#include "olap/schema.h"
+#include "vec/columns/column.h"
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_map.h"
+#include "vec/columns/columns_number.h"
+#include "vec/core/field.h"
+#include "vec/core/sort_block.h"
+#include "vec/core/sort_description.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_map.h"
+
+// this test is gonna to be a column test template for all column which should
make ut test to coverage the function defined in column
+// for example column_array should test this function:
+// size, reserve, resize, empty, byte_size, allocated_bytes, clone_resized,
+// get_shrinked_column, filter, filter_by_selector, serialize_vec,
deserialize_vec, get_max_row_byte_size
+//
+namespace doris::vectorized {
+
+class CommonColumnTest : public ::testing::Test {
+public:
+void SetUp() override {
+col_str = ColumnString::create();
+col_str->insert_data("aaa", 3);
+col_str->insert_data("bb", 2);
+col_str->insert_data("", 4);
+
+col_int = ColumnInt64::create();
+col_int->insert_value(1);
+col_int->insert_value(2);
+col_int->insert_value(3);
+
+col_dcm = ColumnDecimal64::create(0, 3);
+col_dcm->insert_value(1.23);
+col_dcm->insert_value(4.56);
+col_dcm->insert_value(7.89);
+
+col_arr = ColumnArray::create(ColumnInt64::create(),
ColumnArray::ColumnOffsets::create());
+Array array1 = {1, 2, 3};
+Array array2 = {4};
+col_arr->insert(array1);
+col_arr->insert(Array());
+col_arr->insert(array2);
+
+col_map = ColumnMap::create(ColumnString::create(),
ColumnInt64::create(),
+ColumnArray::ColumnOffsets::create());
+Array k1 = {"a", "b", "c"};
+Array v1 = {1, 2, 3};
+Array k2 = {"d"};
+Array v2 = {4};
+Array a = Array();
+Map map1, map2, map3;
+map1.push_back(k1);
+map1.push_back(v1);
+col_map->insert(map1);
+map3.push_back(a);
+map3.push_back(a);
+col_map->insert(map3);
+map2.push_back(k2);
+map2.push_back(v2);
+col_map->insert(map2);
+}
+
+ColumnString::MutablePtr col_str;
+ColumnInt64::MutablePtr col_int;
+ColumnDecimal64::MutablePtr col_dcm;
+ColumnArray::MutablePtr col_arr;
+ColumnMap::MutablePtr col_map;
+
+void checkColumn(const IColumn& col1, const IColumn& col2, const
IDataType& dataType,
Review Comment:
warning: function 'checkColumn' has cognitive complexity of 75 (threshold
50) [readability-function-cognitive-complexity]
```cpp
void checkColumn(const IColumn& col1, const IColumn& col2, const
IDataType& dataType,
^
```
Additional context
**be/test/vec/columns/common_column_test.h:93:** +1, including nesting
penalty of 0, nesting level increased to 1
```cpp
if (WhichDataType(dataType).is_map()) {
^
```
**be/test/vec/columns/common_column_test.h:101:** +1, nesting level
increased to 1
```cpp
} else {
^
```
**be/test/vec/columns/common_column_test.h:102:** +2, including nesting
penalty of 1, nesting level increased to 2
```cpp
if (WhichDataType(dataType).is_int8()) {
^
```
**be/test/vec/columns/common_column_test.h:105:** +3, including nesting
penalty of 2, nesting level increased to 3
```cpp
for (size_t i = 0; i < column_size; ++i) {
^
```
**be/test/vec/columns/common_column_test.h:108:** +1, nesting level
increased to 2
```cpp
} else if (WhichDataType(dataType).is_int16()) {
^
```
**be/test/vec/columns/common_column_test.h:111:** +3, including nesting
penal
Re: [PR] [Improve](columns) update columns and ut [doris]
github-actions[bot] commented on code in PR #42269:
URL: https://github.com/apache/doris/pull/42269#discussion_r1817894691
##
be/test/vec/columns/common_column_test.cpp:
##
@@ -0,0 +1,153 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "vec/columns/common_column_test.h"
+
+#include
+#include
+#include
+
+namespace doris::vectorized {
+
+// MOCK SITUATION TEST -- here ut will test the common column function for
column type, this function called in mocked situation to test multiple column
type.
+TEST_F(CommonColumnTest, SeDeserializeWithArena) {
+MutableColumns columns(4);
+columns[0] = col_str->clone();
+columns[1] = col_int->clone();
+columns[2] = col_arr->clone();
+columns[3] = col_map->clone();
+DataTypes data_types = {std::make_shared(),
std::make_shared(),
+
std::make_shared(std::make_shared()),
+
std::make_shared(std::make_shared(),
+
std::make_shared())};
+ser_deserialize_with_arena_impl(columns, data_types);
+}
+
+TEST_F(CommonColumnTest, SeDeserializeVec) {
+MutableColumns columns(2);
+columns[0] = col_str->clone();
+columns[1] = col_int->clone();
+// array | map | struct get_max_row_byte_size does not implement
+//columns[2] = col_arr->clone();
+//columns[3] = col_map->clone();
+ser_deser_vec(columns, {std::make_shared(),
std::make_shared()});
+}
+
+TEST_F(CommonColumnTest, FilterBySelector) {
+// make a PredictColumn
+auto ptr =
Schema::get_predicate_column_ptr(FieldType::OLAP_FIELD_TYPE_BIGINT, false,
+ReaderType::READER_QUERY);
+auto dt = std::make_shared();
+// 1. 空选择器
+std::vector selector_empty = {};
+filterBySelectorAssert(ptr->get_ptr(), selector_empty, *dt,
col_int->get_ptr(), 0);
+
+// 2. 全选
+std::vector selector_all = {1, 1, 1, 1, 1};
+filterBySelectorAssert(ptr->get_ptr(), selector_all, *dt,
col_int->get_ptr(), 0);
+
+// 3. 全不选
+std::vector select_none = {0, 0, 0, 0, 0};
+filterBySelectorAssert(ptr->get_ptr(), select_none, *dt,
col_int->get_ptr(), 0);
+
+// 4. 部分选择
+std::vector selector_partial = {1, 0, 1, 0, 1};
+filterBySelectorAssert(ptr->get_ptr(), selector_partial, *dt,
col_int->get_ptr(), 0);
+
+// 5. 选择器长度
+// 不匹配
+std::vector selector_invalid = {1, 1, 1, 1};
+filterBySelectorAssert(ptr->get_ptr(), selector_invalid, *dt,
col_int->get_ptr(), 0);
+}
+
+TEST_F(CommonColumnTest, Permute) {
+// 1. generate same rows of columns
+auto columnInt64ValueGetter = [](size_t range_index, size_t
index_in_range) {
+return Field(static_cast(range_index * index_in_range));
+};
+
+auto columnFloat64ValueGetter = [](size_t range_index, size_t
index_in_range) -> Field {
+if (range_index % 2 == 0 && index_in_range % 4 == 0) {
+// quiet_NaN 初始化浮点数，以表明该值当前无效或者尚未定义，
+// 并且不会在传递该值时触发错误。程序可以在之后检查这些值是否为 NaN 来决定下一步的操作。
+return std::numeric_limits::quiet_NaN();
+} else if (range_index % 2 == 0 && index_in_range % 5 == 0) {
+// 负无穷大
+return -std::numeric_limits::infinity();
+} else if (range_index % 2 == 0 && index_in_range % 6 == 0) {
+// 正无穷大
+return std::numeric_limits::infinity();
+}
+Float64 value = static_cast(range_index * index_in_range);
+return Field(value);
+};
+
+auto columnDecimal64ValueGetter = [](size_t range_index, size_t
index_in_range) -> Field {
+Decimal64 val = static_cast(range_index * index_in_range);
+return DecimalField(val, 2);
+};
+
+auto columnStringGetter = [](size_t range_index, size_t index_in_range) ->
Field {
+return Field(std::to_string(range_index * index_in_range));
+};
+ColumnString::MutablePtr col_s = ColumnString::create();
+ColumnInt64::MutablePtr col_i = ColumnInt64::create();
+ColumnFloat64::MutablePtr col_f = ColumnFloat64::create();
+ColumnDecimal64::MutablePtr col_d = ColumnDecimal64::create(0, 2);
+MutableColumns columns;
+columns.emplace_back
Re: [PR] [Improve](columns) update columns and ut [doris]
github-actions[bot] commented on code in PR #42269:
URL: https://github.com/apache/doris/pull/42269#discussion_r1810521581
##
be/src/vec/columns/column_array.cpp:
##
@@ -535,12 +539,23 @@
return filter_number(filt, result_size_hint);
if (typeid_cast(data.get()))
return filter_number(filt, result_size_hint);
+if (typeid_cast(data.get()))
+return filter_number(filt, result_size_hint);
if (typeid_cast(data.get()))
return filter_number(filt, result_size_hint);
if (typeid_cast(data.get()))
return filter_number(filt, result_size_hint);
+if (typeid_cast(data.get()))
+return filter_number(filt, result_size_hint);
+if (typeid_cast(data.get()))
+return filter_number(filt, result_size_hint);
+if (typeid_cast(data.get()))
+return filter_number(filt, result_size_hint);
+if (typeid_cast(data.get()))
+return filter_number(filt, result_size_hint);
Review Comment:
warning: statement should be inside braces
[readability-braces-around-statements]
```suggestion
if (typeid_cast(data.get())) {
return filter_number(filt, result_size_hint);
}
```
##
be/src/vec/columns/column_array.cpp:
##
@@ -527,6 +529,8 @@ ColumnPtr ColumnArray::filter(const Filter& filt, ssize_t
result_size_hint) cons
return filter_number(filt, result_size_hint);
if (typeid_cast(data.get()))
return filter_number(filt, result_size_hint);
+if (typeid_cast(data.get()))
+return filter_number(filt, result_size_hint);
Review Comment:
warning: statement should be inside braces
[readability-braces-around-statements]
```suggestion
if (typeid_cast(data.get())) {
return filter_number(filt, result_size_hint);
}
```
##
be/src/vec/columns/column_array.cpp:
##
@@ -535,12 +539,23 @@
return filter_number(filt, result_size_hint);
if (typeid_cast(data.get()))
return filter_number(filt, result_size_hint);
+if (typeid_cast(data.get()))
+return filter_number(filt, result_size_hint);
if (typeid_cast(data.get()))
return filter_number(filt, result_size_hint);
if (typeid_cast(data.get()))
return filter_number(filt, result_size_hint);
+if (typeid_cast(data.get()))
+return filter_number(filt, result_size_hint);
+if (typeid_cast(data.get()))
+return filter_number(filt, result_size_hint);
+if (typeid_cast(data.get()))
+return filter_number(filt, result_size_hint);
+if (typeid_cast(data.get()))
+return filter_number(filt, result_size_hint);
+if (typeid_cast(data.get()))
+return filter_number(filt, result_size_hint);
Review Comment:
warning: statement should be inside braces
[readability-braces-around-statements]
```suggestion
if (typeid_cast(data.get())) {
return filter_number(filt, result_size_hint);
}
```
##
be/src/vec/columns/column_array.cpp:
##
@@ -855,6 +866,14 @@
// keep ColumnUInt8 for ColumnNullable::null_map
if (typeid_cast(data.get()))
return replicate_number(replicate_offsets);
+if (typeid_cast(data.get()))
+return replicate_number(replicate_offsets);
+if (typeid_cast(data.get()))
+return replicate_number(replicate_offsets);
Review Comment:
warning: statement should be inside braces
[readability-braces-around-statements]
```suggestion
if (typeid_cast(data.get())) {
return replicate_number(replicate_offsets);
}
```
##
be/test/vec/columns/common_column_test.cpp:
##
@@ -0,0 +1,645 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include
+#include
+#include
+
+#include "olap/schema.h"
+#include "runtime/define_primitive_type.h"
+#include "vec/columns/column.h"
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_map.h"
+#include "vec/columns/column_nullable.h"
+#include "vec/columns/columns_number.h"
+#include "vec/columns/predicate_column.h"
+#include "vec/core/field.h"
+#include "vec/core/sort_block.h"
+#include "vec/core/sort_description.h"
+#
Re: [PR] [Improve](columns) update columns and ut [doris]
doris-robot commented on PR #42269: URL: https://github.com/apache/doris/pull/42269#issuecomment-2428982630 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
