317brian commented on code in PR #14600: URL: https://github.com/apache/druid/pull/14600#discussion_r1287727569
########## docs/do-not-merge.md: ########## @@ -182,163 +194,140 @@ Updated EARLIEST_BY and LATEST_BY function signatures as follows: * Changed `EARLIEST(expr, timeColumn)` to `EARLIEST_BY(expr, timeColumn)` * Changed `LATEST(expr, timeColumn)` to `LATEST_BY(expr, timeColumn)` -[14352](https://github.com/apache/druid/pull/14352) +[#14352](https://github.com/apache/druid/pull/14352) +### INFORMATION_SCHEMA.ROUTINES TABLE -### Exposed Druid functions +Use the new `INFORMATION_SCHEMA.ROUTINES` to programmatically get information about the functions that Druid SQL supports. -Exposed information about Druid functions in the `INFORMATION_SCHEMA.ROUTINES` table. The table contains the following columns: +For more information, such as the available columns, see [ROUTINES table](https://druid.apache.org/docs/latest/querying/sql-metadata-tables.html#routines-table) -* ROUTINE_CATALOG: Name of the database. Always set to druid. -* ROUTINE_SCHEMA: Name of the schema. Always set to INFORMATION_SCHEMA. -* ROUTINE_NAME: Name of the function or operator. For example: APPROX_COUNT_DISTINCT_DS_THETA. -* ROUTINE_TYPE: Type of routine. Always set to FUNCTION. -* IS_AGGREGATOR: Determines if the routine is an aggregator. Returns YES for aggregator functions; NO for scalar functions. -* SIGNATURES: Possible signatures for the routine as a string. - -[14378](https://github.com/apache/druid/pull/14378) +[#14378](https://github.com/apache/druid/pull/14378) ### New Broker configuration for SQL schema migrations -A new broker configuration, `druid.sql.planner.metadataColumnTypeMergePolicy`, adds configurable modes to how column types are computed for the SQL table schema when faced with differences between segments. +You can now better control how Druid reacts to schema changes between segments. This can make querying more resilient when newer segments introduce different types, such as if a column previously contained LONG values and newer segments contain STRING. -The new `leastRestrictive` mode chooses the most appropriate type that data across all segments can best be coerced into and is the default behavior. This is a subtle behavior change around when segment driven schema migrations will take effect for the SQL schema: +Use the new Broker configuration, `druid.sql.planner.metadataColumnTypeMergePolicy` to control how column types are computed for the SQL table schema when faced with differences between segments. -- With `leastRestrictive` mode, the schema only updates once all segments are reindexed to the new type. -- With `latestInterval` mode, the SQL schema gets updated as soon as the first job with the new schema publishes segments in the latest time interval of the data. +Set it to one of the following: +- `leastRestrictive`: the schema only updates once all segments are reindexed to the new type. +- `latestInterval`: the SQL schema gets updated as soon as the first job with the new schema publishes segments in the latest time interval of the data. + `leastRestrictive` can have better query time behavior and eliminates some query time errors that can occur when using `latestInterval`. -[14319](https://github.com/apache/druid/pull/14319) +[#14319](https://github.com/apache/druid/pull/14319) ### EXPLAIN PLAN improvements The EXPLAIN PLAN result includes a new column `ATTRIBUTES` that describes the attributes of a query. -See [SQL query translation](docs/querying/sql-translation.md) for more information and examples. -[14391](https://github.com/apache/druid/pull/14391) +For more information, see [SQL translation](https://druid.apache.org/docs/latest/querying/sql-translation.html) +[#14391](https://github.com/apache/druid/pull/14391) -## Metrics and monitoring +### New query filters -### Added a new SysMonitorOshi monitor +TBD -Added a new monitor `SysMonitorOshi` to replace `SysMonitor`. The new monitor has a wider support for different machine architectures including ARM instances. We recommend switching to `SysMonitorOshi` as `SysMonitor` is now deprecated and will be removed in future releases. +## Metrics and monitoring -[14359](https://github.com/apache/druid/pull/14359) ### Monitor for Overlord and Coordinator service health Added a new monitor `ServiceStatusMonitor` to monitor the service health of the Overlord and Coordinator. -[14443](https://github.com/apache/druid/pull/14443) +[#14443](https://github.com/apache/druid/pull/14443) + +### New Broker metrics -### New segment metrics +TBD for name changes The following metrics are now available for Brokers: -|Metric|Description|Dimensions|Normal value| -|------|-----------|----------|------------| +|Metric|Description|Dimensions| +|------|-----------|----------| |`segment/metadatacache/refresh/count`|Number of segments to refresh in broker segment metadata cache. Emitted once per refresh per datasource.|`dataSource`| | |`segment/metadatacache/refresh/time`|Time taken to refresh segments in broker segment metadata cache. Emitted once per refresh per datasource.|`dataSource`| | + +[#14453](https://github.com/apache/druid/pull/14453) + +### New Coordinator metrics + +|Metric|Description|Dimensions|Normal value| +|------|-----------|----------| --| | `segment/loadQueue/assigned` | Number of segments assigned for load or drop to the load queue of a server. |`dataSource`,`server` | Varies | | `segment/loadQueue/success` |Number of segment assignments that completed successfully.|`dataSource`, `server`|Varies| | `segment/loadQueue/cancelled` |Number of segment assignments that were canceled before completion. |`dataSource`,`server` | 0 | | `segment/loadQueue/failed` |Number of segment assignments that failed to complete.|`dataSource`, `server`|0| -[14453](https://github.com/apache/druid/pull/14453) -[13197](https://github.com/apache/druid/pull/13197) +[#13197](https://github.com/apache/druid/pull/13197) ### New metrics for task completion updates -The following metrics have been added: -* `task/status/queue/count`: Monitors the number of queued items -* `task/status/updated/count`: Monitors the number of processed items +| Metric | Description | Normal value | +| ------ | ----------- | ---------- | +| `task/status/queue/count` | Monitors the number of queued items | Varies | +| `task/status/updated/count` | Monitors the number of processed items | Varies | -[14533](https://github.com/apache/druid/pull/14533) +[#14533](https://github.com/apache/druid/pull/14533) ### Added `groupId` to Overlord task metrics Added `groupId` to task metrics emitted by the Overlord. This is helpful for grouping metrics like task/run/time by a single task group, such as a single compaction task or a single MSQ query. -[14402](https://github.com/apache/druid/pull/14402) +[#14402](https://github.com/apache/druid/pull/14402) ### New metrics for monitoring sync status of `HttpServerInventoryView` +TBD for name change + |Metric|Description|Dimensions|Normal value| |------|-----------|-----------|----------| -|`segment/serverview/sync/healthy`|Sync status of the Coordinator/Broker with a segment-loading server such as a Historical or Peon. Emitted by the Coordinator and Brokers only when [HTTP-based server view](../configuration/index.md#segment-management) is enabled. This metric can be used in conjunction with `segment/serverview/sync/unstableTime` to debug slow startup of the Coordinator.|`server`, `tier`|1 for fully synced servers, 0 otherwise| -|`segment/serverview/sync/unstableTime`|Time in milliseconds for which the Coordinator/Broker has been failing to sync with a segment-loading server. Emitted by the Coordinator and Brokers only when [HTTP-based server view](../configuration/index.md#segment-management) is enabled.|`server`, `tier`|Not emitted for synced servers.| -## Cluster management - -### Removed unused Coordinator dynamic configuration properties +|`segment/serverview/sync/healthy`|Sync status of the Coordinator/Broker with a segment-loading server such as a Historical or Peon. Emitted by the Coordinator and Brokers only when HTTP-based server view is enabled. This metric can be used in conjunction with `segment/serverview/sync/unstableTime` to debug slow startup of the Coordinator.|`server`, `tier`|1 for fully synced servers, 0 otherwise| +|`segment/serverview/sync/unstableTime`|Time in milliseconds for which the Coordinator/Broker has been failing to sync with a segment-loading server. Emitted by the Coordinator and Brokers only when HTTP-based server view is enabled.|`server`, `tier`|Not emitted for synced servers.| Review Comment: Manually updated in https://github.com/apache/druid/pull/14600/commits/0959b288c5a56b65cd0394f4d6f4be1b810a31ff -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
