yiguolei opened a new issue, #58605: URL: https://github.com/apache/doris/issues/58605
# New Feature ## AI & Search - Inverted index supports custom analyzers, including Pinyin tokenizer and Pinyin filter (#57097) - Added support for multi-position PhraseQuery in inverted index search functions (#57588) - Added Ann index only-scan capability (#57243) ## Function - Added the sem aggregate function (#57545) - Supported the factorial simple SQL function derived from Hive (#57144) - Added support for zero-width assertions in some regular expression functions (#57643) - Enabled GROUP BY and DISTINCT operations for JSON type (#57679) - Added the add_time/sub_time time functions (#56200) - Added the deduplicate_map function (#58403) ## Materialized View (MTMV) - Materialized views can still participate in transparent query rewrite when data changes occur in their non-partitioned base tables (#56745) - Supported creating MTMV based on views (#56423) - MTMV refresh supports multiple PCT tables (#58140) - Supported window function rewrite when materialized views contain window functions (#55066) ## Data Lake - Implemented the Iceberg rewrite_data_files action to support table optimization and compaction (#56413, #56638) - Supported VARBINARY type mapping for Hive/Iceberg/Paimon/JDBC tables (#57821, #58215) - Supported Partition Evolution DDL for Iceberg tables (#57972) # Optimizations - Optimized the performance of the FROM_UNIXTIME function (#57423) - Removed the castTo conversion in PartitionKey comparison to improve partition processing efficiency (#57518) - Optimized the performance of Parquet reader when decoding RLE_DICTIONARY encoding (#57208) - Reduced the memory footprint of the Column class in Catalog (#57401) - Accumulated multiple small batches before Ann index training to improve training efficiency (#57623) # Bug Fixes ## Query - Fixed the issue where the utc_time function returned incorrect results when the input was null (#57716) - Fixed the exception thrown when UNION ALL is combined with TVF (#57889) - Fixed the problem that the WHERE clause contained non-key columns when creating a materialized view on a unique key table (#57915) - Fixed window functions: enabled constant expression evaluation for the offset parameter of LAG/LEAD (#58200) - Fixed aggregate functions: abnormal push-down of aggregate operations before projection on nullable columns; count push-down aggregation issue on non-null columns (#58234) - Fixed time functions: the second/microsecond functions did not handle time literals; time_to_sec reported errors due to garbage values when processing null values (#56659, #58410) - Fixed AI functions: unknown error occurred when _exec_plan_fragment_impl called AI functions (#58521) - Fixed geo module: memory leak in the geo module (#58004) - Fixed information_schema: timezone format incompatibility when using offset timezone (#58412) ## Materialized View and Schema Change - Fixed the failure of rewrite when materialized views contain group sets and filters above scan (#57343) - Fixed the coredump issue caused by reading non-overlapping segments from a single rowset during heavy schema change (#57191) ## Storage-Compute Separation - Fixed the issue of broadcast remote read in TopN queries (#58044) - Fixed the accumulation of tablet deletion tasks in the cloud environment (#58131) - Fixed the problem of long service startup time during the first boot in the cloud environment (#58152) ## Data Lake - Fixed Iceberg: enabled dynamic partition pruning only for identity partitions (#58033) - Fixed the permission authentication issue when loading Iceberg partitions (#57988) - Fixed the partition path scheme mismatch when inserting into Hive partitioned tables on object storage (#57973) - Fixed the issue where Hive cache was not refreshed when inconsistent (#58074) - Fixed Paimon Catalog: OSS access failure when using DLS endpoint (#58099) - Fixed Iceberg: FE did not refresh logs after ALTER TABLE ... EXECUTE; enabled dynamic partition pruning only for identity partitions; added auth regression tests for Iceberg system tables (#58355, #58033, #58298) - Fixed Hive: StackOverflowError caused by insert overwrite on S3-compatible storage; Hive cache not refreshed when inconsistent; partition path scheme mismatch (#58504, #58074, #57973) - Fixed Paimon: OSS access failure under DLS endpoint; supported user-defined S3 config prefixes and unified to HDFS S3A protocol (#58099, #57116) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
