317brian commented on code in PR #18630: URL: https://github.com/apache/druid/pull/18630#discussion_r2427273191
########## docs/release-info/release-notes.md: ########## @@ -65,38 +115,205 @@ This section contains detailed release notes separated by areas. #### Other web console improvements +- Added better support for MSQ task engine-based compaction tasks. They now use the stages pane to render the compaction report instead of showing the JSON [#18545](https://github.com/apache/druid/pull/18545) +- Added a version column to the **Services** tab so that you can see what version a service is running. This is helpful during rolling upgrades to verify the state of the cluster and upgrade [#18542](https://github.com/apache/druid/pull/18542) +- Improved the resiliency of the web console when the supervisor history is extensive [#18416](https://github.com/apache/druid/pull/18416) + + ### Ingestion +- Improved task performance. Druid now polls from memory before fetching task information from the metadata database [#18448](https://github.com/apache/druid/pull/18448) +- Improved task execution so that they can successfully complete even if there are problems pushing logs and reports to deep storage [#18210](https://github.com/apache/druid/pull/18210) + #### SQL-based ingestion ##### Other SQL-based ingestion improvements +- Added the ability to configure the maximum frazme size. Generally, you don't need to change this unless you have very large rows [#18442](https://github.com/apache/druid/pull/18442) +- Added logging for when segment processing fails [#18378](https://github.com/apache/druid/pull/18378) +- Improved logging to store the cause of invalid field exceptions [#18517](https://github.com/apache/druid/pull/18517) [#18517](https://github.com/apache/druid/pull/18517) + + #### Streaming ingestion ##### Other streaming ingestion improvements +- Added a count parameter to the supervisor history API [#18416](https://github.com/apache/druid/pull/18416) + ### Querying +#### Caching scan query results + +Druid now supports result-level caching for scan queries. + +By default, this behavior is turned off. To enable, override the `druid.*.cache.unCacheable` property. + +[#18568](https://github.com/apache/druid/pull/18568) + +#### New expressions for sketches + +Druid now supports the following expressions: + +- `HLL_SKETCH_ESTIMATE_WITH_ERROR_BOUNDS` +- `THETA_SKETCH_ESTIMATE_WITH_ERROR_BOUNDS` + +These estimates work on a sketch column and have the same behavior as the post aggregators. + +[#18426](https://github.com/apache/druid/pull/18426) + +#### New multi-value SQL functions + +Druid now supports the following multi-value functions: + +- `MV_FILTER_REGEX`: filters a multi-value expression to include only values matching the specified regular expression pattern +- `MV_FILTER_PREFIX`: filters a multi-value expression to include only values that start with the specified prefix + +[#18281](https://github.com/apache/druid/pull/18281) + +#### Exception handling + +You can now write exceptions that Druid encounters as a row and then verify whether the exception is expected. To use this feature, set the `writeExceptionBodyAsResponseRow` query context parameter to `true`. + +[#18571](https://github.com/apache/druid/pull/18571) + #### Other querying improvements +- Added stricter validation for `GREATEST` and `LEAST` [#18562](https://github.com/apache/druid/pull/18562) +- Added a query context option called `realtimeSegmentsOnly` that returns results from realtime segments only when set to `true` [#18329](https://github.com/apache/druid/pull/18329) +- Improved the performance of the following query types by vectorizing them: + - `CASE_SEARCHED` and `CASE_SIMPLE` queries [#18512](https://github.com/apache/druid/pull/18512) + - `timestamp_ceil` and `timestamp_extract` queries [#18517](https://github.com/apache/druid/pull/18517) + - `IF` expressions where the `THEN` and `ELSE` expressions [#18507](https://github.com/apache/druid/pull/18507) +- Improved projections so that the granularty in queries can match UTC time zones [#18403](https://github.com/apache/druid/pull/18403) +- Improved the deserialization time for intervals by more than 40% [#18477](https://github.com/apache/druid/pull/18477) +- Improved the performance of `AND`/`OR` as well as `NOT`/`IS TRUE`/`IS FALSE` [#18491](https://github.com/apache/druid/pull/18491) [#18488](https://github.com/apache/druid/pull/18488) +- Improved the performance of scan queries [#18441](https://github.com/apache/druid/pull/18441) +- Improved the performance of metadata queries by parallelizing them at the data-node level [#18592](https://github.com/apache/druid/pull/18592) + ### Cluster management +#### Error message strategy + +You can now configure Druid to log an error message and return an error ID for non-user targeted messages. This can be configured by setting `druid.server.http.errorResponseTransform.strategy` to `persona`. + +[#18487](https://github.com/apache/druid/pull/18487) + +#### Kerberos authentication + +The `druid.auth.authenticator.kerberos.cookieSignatureSecret` config is now mandatory. + +[#18368](https://github.com/apache/druid/pull/18368) + #### Other cluster management improvements +- Added a `version` column `sys.server`. This is useful during rolling upgrades to verify the state of the cluster [#18542](https://github.com/apache/druid/pull/18542) +- Added support for short, unique index names in metadata stores [#18515](https://github.com/apache/druid/pull/18515) +- Added support for proportional `stopTaskCount` to the task count autoscaler +- Added headers to the user agent for services, improving traceability between services [#18505](https://github.com/apache/druid/pull/18505) +- Added a new Kafka emitter config: `druid.emitter.kafka.producer.shutdownTimeout`. It controls how long the Kafka producer waits for pending requests to finish before shutting down [#18427](https://github.com/apache/druid/pull/18427) +- Changed how Netty worker threads get calculated, lowering the number of threads used at small processor counts [#18493](https://github.com/apache/druid/pull/18493) +- Changed the response for the `/status/properties` to be alphabetical. The response is more readable now and related keys are grouped together [#18506](https://github.com/apache/druid/pull/18506) +- Improved segment loading during startup in certain situations by optimizing concurrent writing [#18470](https://github.com/apache/druid/pull/18470) + ### Data management #### Other data management improvements +- Storing segments in S3 now supports not using zip compression, like local storage [#18544](https://github.com/apache/druid/pull/18544) Review Comment: ```suggestion - Improved S3 storage to support storing segments in S3 without zip compression [#18544](https://github.com/apache/druid/pull/18544) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
