wangbo opened a new issue #1943: Segment v2 use string real length URL: https://github.com/apache/incubator-doris/issues/1943 Current segment v2 use fixed length to allocate string field,using real length of string field is better # major process 1. init a Field by type in SegmentWriter.init,then ColumnWriter and ColumnZoneMapBuilder holds it instead of holding a typeinfo,this is because field contains whole info of a specific type Field,while typeInfo only contains method 2. using polymorphism to realize string type allocate value with length while other type allocate value with default method. Char/VarcharField override allocate_value_from_arena,and Char/VarcharField contains string real length # Other question 1. bug of set_to_max; current set_to_max of varchar,hard code setting size of varchar length to 1,but it should use user define varchar length. we can use field.length(come from tabletColumn.length) to set_to_max 2. problem of using field.length in set_to_max segment v1 think field.length=sizeof(StringLengthType) + slice.size segment v2 think field.length=slice.size so confict happends I defined a new method signature (set_to_max_v1)to let segment v1 code using it. the logic of set_to_max_v1 not changed,it is still hard code. segment v2 using set_to_max method, and it set string using string'real length 3. current code refers field.set_to_max wrapper_field.cpp, change to set_to_max_v1 stream_index_common.cpp,no change,string and hll skipped row_cursor.cpp, change to set_to_max_v1
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
