clintropolis commented on code in PR #19945: URL: https://github.com/apache/druid/pull/19945#discussion_r4152507694
########## docs/release-info/release-notes.md: ########## @@ -57,6 +57,80 @@ For tips about how to write a good release note, see [Release notes](https://git This section contains important information about new and existing features. +#### Java + +Druid now supports Java 25. While Java 21 is still supported, we recommend you upgrade to Java 25. + +Support for Java 17 has been dropped. + +[#19304](https://github.com/apache/druid/pull/19304) [#19336](https://github.com/apache/druid/pull/19336) + +#### Historical tier aliases + +You can map a virtual tier name to a number of real Historical tiers with the `historicalTierAliases` Coordinator dynamic configuration. This creates a single identifier for a group of Historical tiers. When a load/drop rule references the alias, the Coordinator replaces it with the actual tiers. For example, if you map the Historical tiers `hot_1` and `hot_2` to the alias `hot`, the rule `{"hot": 2}` loads 2 replicas of each onto `hot_1` and `hot_2`. + +[#19204](https://github.com/apache/druid/pull/19204) [#19667](https://github.com/apache/druid/pull/19667) + +#### New load rule types + +Adds a new family of retention rules, `loadPartialByPeriod`, `loadPartialByInterval`, `loadPartialForever`, laying the groundwork for partial loading of version 10 segment projections on Historicals. + +[#19374](https://github.com/apache/druid/pull/19374) + +#### Realtime segments mode query context + +The `realtimeSegmentsOnly` query context parameter has been deprecated and replaced with `realtimeSegmentsMode`. + +You can set `realtimeSegmentsMode` to one of the following: + +- `include` (default): query all segments, including realtime segments +- `exclude`: skip realtime segments for the query +- `exclusive`: query only realtime segments + +This is useful when performing things like blue/green deployments and you only want to query new Historical replica autoscaling groups and not touch any "live" nodes (neither realtime nor historical). + +[#19486](https://github.com/apache/druid/pull/19486) + +#### Faster segment metadata search + +You can set `druid.segment.timeline.fastIntervalSearch` to `true` so that Druid uses an index based on interval trees to store that metadata in memory for faster identification and retrieval. + +This feature is off by default. + +[#19138](https://github.com/apache/druid/issues/19138) [#19850](https://github.com/apache/druid/issues/19850) + +#### Fetching files in the background + +The MSQ task engine now supports workers fetching input files from cloud storage asynchronously. This overlaps downloading with processing, which generally improves throughput when reading larger numbers of files. Otherwise, workers stream each file directly from cloud storage while processing it. + +This feature is on by default and is controlled by the `backgroundFetchExternalFiles` setting. + +[#19539](https://github.com/apache/druid/pull/19539) + +#### Segment prefetching for Dart + +Dart now supports the runtime property `druid.msq.dart.worker.segmentLoadAheadCount`, which controls the number of segments that Dart prefetches. If set greater than 0 for a worker, this setting becomes the default `segmentLoadAheadCount` value for the worker. If a query includes the `segmentLoadAheadCount` query context parameter, the query context takes precedence. + +[#19559](https://github.com/apache/druid/pull/19559) + +#### Partial segment loading Review Comment: partial load rules are actually separate from the linked PRs (there are a handful of those PRs too). 19535 is about allowing historicals running in virtual storage mode, with v10 segments which are stored in deep storage without zip, to load on demand at the column level at query time (so dictated by what the query needs to read rather than load rules), when using Dart or MSQ (legacy engine still requires full download up front in virtual mode). The storage accounting of the historical cache is done at the 'bundle' level (base table, projections, cluster groups), where the cache has an entry for each bundle and we just lazy load columns contained in that bundle into it as the query needs. Partial load rules work with this partial load functionality by allowing virtual mode historicals to eagerly load parts of segments at the 'bundle' level (base table, projections, cluster groups) and make those bundles sticky so that they cannot be evicted/are always 'hot' even if the rest of the segment can be loaded and evicted on demand. All of this stuff is quite experimental, and will probably be in a lot nicer state next release. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
