vtlim commented on code in PR #14623: URL: https://github.com/apache/druid/pull/14623#discussion_r1291786722
########## docs/api-reference/retention-rules-api.md: ########## @@ -23,47 +23,487 @@ sidebar_label: Retention rules ~ under the License. --> -This document describes the API endpoints for managing retention rules in Apache Druid. +This topic describes the API endpoints for managing retention rules in Apache Druid. You can configure retention rules in the Druid web console or API. -## Retention rules +Druid uses retention rules to determine what data is retained in the cluster. Druid supports load, drop, and broadcast rules. See [using rules to drop and retain data](../operations/rule-configuration.md) for more information. Review Comment: ```suggestion Druid uses retention rules to determine what data is retained in the cluster. Druid supports load, drop, and broadcast rules. See [Using rules to drop and retain data](../operations/rule-configuration.md) for more information. ``` ########## docs/api-reference/retention-rules-api.md: ########## @@ -23,47 +23,487 @@ sidebar_label: Retention rules ~ under the License. --> -This document describes the API endpoints for managing retention rules in Apache Druid. +This topic describes the API endpoints for managing retention rules in Apache Druid. You can configure retention rules in the Druid web console or API. -## Retention rules +Druid uses retention rules to determine what data is retained in the cluster. Druid supports load, drop, and broadcast rules. See [using rules to drop and retain data](../operations/rule-configuration.md) for more information. -Note that all _interval_ URL parameters are ISO 8601 strings delimited by a `_` instead of a `/` as in `2016-06-27_2016-06-28`. +In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments. -`GET /druid/coordinator/v1/rules` +## Update retention rules for a datasource -Returns all rules as JSON objects for all datasources in the cluster including the default datasource. +Update one or more retention rules for a datasource. Retention rules can be submitted as an array of rule objects in the request body and overwrite any existing rules for the datasource. Rules are read in the order in which they appear, see [rule structure](../operations/rule-configuration.md) for more information. -`GET /druid/coordinator/v1/rules/{dataSourceName}` +Note that this endpoint returns an `HTTP 200` even if the datasource does not exist. -Returns all rules for a specified datasource. +### URL -`GET /druid/coordinator/v1/rules/{dataSourceName}?full` +<code class="postAPI">POST</code> <code>/druid/coordinator/v1/rules/:datasource</code> -Returns all rules for a specified datasource and includes default datasource. +### Header parameters -`GET /druid/coordinator/v1/rules/history?interval=<interval>` +The endpoint supports a set of optional header parameters to populate the `author` and `comment` fields in the `auditInfo` property for audit history. -Returns audit history of rules for all datasources. Default value of interval can be specified by setting `druid.audit.manager.auditHistoryMillis` (1 week if not configured) in Coordinator `runtime.properties`. +* `X-Druid-Author` (optional) + * Type: String + * A string representing the author making the configuration change. +* `X-Druid-Comment` (optional) + * Type: String + * A string describing the update. -`GET /druid/coordinator/v1/rules/history?count=<n>` +### Responses -Returns last `n` entries of audit history of rules for all datasources. +<!--DOCUSAURUS_CODE_TABS--> -`GET /druid/coordinator/v1/rules/{dataSourceName}/history?interval=<interval>` +<!--200 SUCCESS--> -Returns audit history of rules for a specified datasource. Default value of interval can be specified by setting `druid.audit.manager.auditHistoryMillis` (1 week if not configured) in Coordinator `runtime.properties`. +*Successfully updated retention rules for specified datasource* -`GET /druid/coordinator/v1/rules/{dataSourceName}/history?count=<n>` +<!--END_DOCUSAURUS_CODE_TABS--> -Returns last `n` entries of audit history of rules for a specified datasource. +--- + +### Sample request + +The following example sets a set of broadcast, load, and drop retention rules for the `kttm1` datasource. + +<!--DOCUSAURUS_CODE_TABS--> + +<!--cURL--> + +```shell +curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/rules/kttm1"; \ +--header 'X-Druid-Author: doc intern' \ +--header 'X-Druid-Comment: submitted via api' \ +--header 'Content-Type: application/json' \ +--data '[ + { + "type": "broadcastForever" + }, + { + "type": "loadForever", + "tieredReplicants": { + "_default_tier": 2 + }, + "useDefaultTierForNull": true + }, + { + "type": "dropByPeriod", + "period": "P1M" + } +]' +``` + +<!--HTTP--> + +```HTTP +POST /druid/coordinator/v1/rules/kttm1 HTTP/1.1 +Host: http://ROUTER_IP:ROUTER_PORT +X-Druid-Author: doc intern +X-Druid-Comment: submitted via api +Content-Type: application/json +Content-Length: 273 + +[ + { + "type": "broadcastForever" + }, + { + "type": "loadForever", + "tieredReplicants": { + "_default_tier": 1 + }, + "useDefaultTierForNull": true + }, + { + "type": "dropByPeriod", + "period": "P1M" + } +] +``` + +<!--END_DOCUSAURUS_CODE_TABS--> + +### Sample response + +A successful request returns an HTTP `200 OK` and an empty response body. + +## Update default retention rules for all datasources + +Update one or more default retention rules for all datasources. Retention rules can be submitted as an array of rule objects in the request body and overwrite any existing rules for the datasource. To remove default retention rules for all datasources, submit an empty rule array in the request body. Rules are read in the order in which they appear, see [rule structure](../operations/rule-configuration.md) for more information. + +### URL + +<code class="postAPI">POST</code> <code>/druid/coordinator/v1/rules/_default</code> + +### Header parameters + +The endpoint supports a set of optional header parameters to populate the `author` and `comment` fields in the `auditInfo` property for audit history. + +* `X-Druid-Author` (optional) + * Type: String + * A string representing the author making the configuration change. +* `X-Druid-Comment` (optional) + * Type: String + * A string describing the update. + +### Responses + +<!--DOCUSAURUS_CODE_TABS--> + +<!--200 SUCCESS--> + +*Successfully updated default retention rules* + +<!--500 SERVER ERROR--> + +*Error with request body* + +<!--END_DOCUSAURUS_CODE_TABS--> + +--- + +### Sample request + +The following example updates the default retention rule for all datasources with a `loadByInterval` rule. + +<!--DOCUSAURUS_CODE_TABS--> + +<!--cURL--> + +```shell +curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/rules/_default"; \ +--header 'Content-Type: application/json' \ +--data '[ + { + "type": "loadByInterval", + "tieredReplicants": {}, + "useDefaultTierForNull": false, + "interval": "2010-01-01/2020-01-01" + } +]' +``` + +<!--HTTP--> + +```HTTP +POST /druid/coordinator/v1/rules/_default HTTP/1.1 +Host: http://ROUTER_IP:ROUTER_PORT +Content-Type: application/json +Content-Length: 205 + +[ + { + "type": "loadByInterval", + "tieredReplicants": {}, + "useDefaultTierForNull": false, + "interval": "2010-01-01/2020-01-01" + } +] +``` + +<!--END_DOCUSAURUS_CODE_TABS--> + +### Sample response + +A successful request returns an HTTP `200 OK` and an empty response body. + +## Get an array of all retention rules + +Retrieves all current retention rules in the cluster including the default retention rule. Returns an array of objects for each datasource and their associated retention rule. + +### URL + +<code class="getAPI">GET</code> <code>/druid/coordinator/v1/rules</code> + +### Responses + +<!--DOCUSAURUS_CODE_TABS--> + +<!--200 SUCCESS--> + +*Successfully retrieved retention rules* + +<!--END_DOCUSAURUS_CODE_TABS--> + +--- + +### Sample request + +<!--DOCUSAURUS_CODE_TABS--> + +<!--cURL--> + +```shell +curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/rules"; +``` + +<!--HTTP--> + +```HTTP +GET /druid/coordinator/v1/rules HTTP/1.1 +Host: http://ROUTER_IP:ROUTER_PORT +``` + +<!--END_DOCUSAURUS_CODE_TABS--> + +### Sample response + +<details> + <summary>Click to show sample response</summary> + + ```json +{ + "_default": [ + { + "tieredReplicants": { + "_default_tier": 2 + }, + "type": "loadForever" + } + ], + "social_media": [ + { + "interval": "2023-01-01T00:00:00.000Z/2023-02-01T00:00:00.000Z", + "type": "dropByInterval" + } + ], + "wikipedia_api": [], +} + ``` +</details> + +## Get an array of retention rules for a datasource + +Retrieves an array of rule objects for a single datasource. Returns an empty array if there are no retention rules. + +Note that this endpoint returns an `HTTP 200` even if the datasource does not exist. + +### URL + +<code class="getAPI">GET</code> <code>/druid/coordinator/v1/rules/:datasource</code> + +### Query parameters + +* `full` (optional) + * Include the default retention rule for the datasource in the response. + +### Responses + +<!--DOCUSAURUS_CODE_TABS--> + +<!--200 SUCCESS--> + +*Successfully retrieved retention rules* + +<!--END_DOCUSAURUS_CODE_TABS--> + +--- + +### Sample request + +The following example retrieves the custom retention rules and default retention rules for datasource with the name `social_media`. + +<!--DOCUSAURUS_CODE_TABS--> + +<!--cURL--> + +```shell +curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/rules/social_media?full=null"; +``` + +<!--HTTP--> + +```HTTP +GET /druid/coordinator/v1/rules/social_media?full=null HTTP/1.1 +Host: http://ROUTER_IP:ROUTER_PORT +``` + +<!--END_DOCUSAURUS_CODE_TABS--> + +### Sample response + +<details> + <summary>Click to show sample response</summary> + + ```json +[ + { + "interval": "2020-01-01T00:00:00.000Z/2022-02-01T00:00:00.000Z", + "type": "dropByInterval" + }, + { + "interval": "2010-01-01T00:00:00.000Z/2020-01-01T00:00:00.000Z", + "tieredReplicants": { + "_default_tier": 2 + }, + "type": "loadByInterval" + }, + { + "tieredReplicants": { + "_default_tier": 2 + }, + "type": "loadForever" + } +] + ``` + +</details> + +## Get audit history for all datasources + +Retrieves the audit history of rules for all datasources over an interval of time. The default value of `interval` can be specified by setting `druid.audit.manager.auditHistoryMillis` (1 week if not configured) in the `runtime.properties` file for the Coordinator. Review Comment: ```suggestion Retrieves the audit history of rules for all datasources over an interval of time. The default interval is 1 week. You can change this period by setting `druid.audit.manager.auditHistoryMillis` in the `runtime.properties` file for the Coordinator. ``` ########## docs/api-reference/retention-rules-api.md: ########## @@ -23,47 +23,487 @@ sidebar_label: Retention rules ~ under the License. --> -This document describes the API endpoints for managing retention rules in Apache Druid. +This topic describes the API endpoints for managing retention rules in Apache Druid. You can configure retention rules in the Druid web console or API. -## Retention rules +Druid uses retention rules to determine what data is retained in the cluster. Druid supports load, drop, and broadcast rules. See [using rules to drop and retain data](../operations/rule-configuration.md) for more information. -Note that all _interval_ URL parameters are ISO 8601 strings delimited by a `_` instead of a `/` as in `2016-06-27_2016-06-28`. +In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments. -`GET /druid/coordinator/v1/rules` +## Update retention rules for a datasource -Returns all rules as JSON objects for all datasources in the cluster including the default datasource. +Update one or more retention rules for a datasource. Retention rules can be submitted as an array of rule objects in the request body and overwrite any existing rules for the datasource. Rules are read in the order in which they appear, see [rule structure](../operations/rule-configuration.md) for more information. Review Comment: Active voice, and link to reduce friction for creating the request ```suggestion Updates one or more retention rules for a datasource. The request body takes an array of retention rule objects. For details on defining retention rules, see the following sources: * [Load rules](../operations/rule-configuration.md#load-rules) * [Drop rules](../operations/rule-configuration.md#drop-rules) * [Broadcast rules](../operations/rule-configuration.md#broadcast-rules) This request overwrites any existing rules for the datasource. Druid reads rules in the order in which they appear; for more information, see [rule structure](../operations/rule-configuration.md). ``` ########## docs/api-reference/retention-rules-api.md: ########## @@ -23,47 +23,487 @@ sidebar_label: Retention rules ~ under the License. --> -This document describes the API endpoints for managing retention rules in Apache Druid. +This topic describes the API endpoints for managing retention rules in Apache Druid. You can configure retention rules in the Druid web console or API. -## Retention rules +Druid uses retention rules to determine what data is retained in the cluster. Druid supports load, drop, and broadcast rules. See [using rules to drop and retain data](../operations/rule-configuration.md) for more information. -Note that all _interval_ URL parameters are ISO 8601 strings delimited by a `_` instead of a `/` as in `2016-06-27_2016-06-28`. +In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments. -`GET /druid/coordinator/v1/rules` +## Update retention rules for a datasource -Returns all rules as JSON objects for all datasources in the cluster including the default datasource. +Update one or more retention rules for a datasource. Retention rules can be submitted as an array of rule objects in the request body and overwrite any existing rules for the datasource. Rules are read in the order in which they appear, see [rule structure](../operations/rule-configuration.md) for more information. -`GET /druid/coordinator/v1/rules/{dataSourceName}` +Note that this endpoint returns an `HTTP 200` even if the datasource does not exist. -Returns all rules for a specified datasource. +### URL -`GET /druid/coordinator/v1/rules/{dataSourceName}?full` +<code class="postAPI">POST</code> <code>/druid/coordinator/v1/rules/:datasource</code> -Returns all rules for a specified datasource and includes default datasource. +### Header parameters -`GET /druid/coordinator/v1/rules/history?interval=<interval>` +The endpoint supports a set of optional header parameters to populate the `author` and `comment` fields in the `auditInfo` property for audit history. -Returns audit history of rules for all datasources. Default value of interval can be specified by setting `druid.audit.manager.auditHistoryMillis` (1 week if not configured) in Coordinator `runtime.properties`. +* `X-Druid-Author` (optional) + * Type: String + * A string representing the author making the configuration change. +* `X-Druid-Comment` (optional) + * Type: String + * A string describing the update. -`GET /druid/coordinator/v1/rules/history?count=<n>` +### Responses -Returns last `n` entries of audit history of rules for all datasources. +<!--DOCUSAURUS_CODE_TABS--> -`GET /druid/coordinator/v1/rules/{dataSourceName}/history?interval=<interval>` +<!--200 SUCCESS--> -Returns audit history of rules for a specified datasource. Default value of interval can be specified by setting `druid.audit.manager.auditHistoryMillis` (1 week if not configured) in Coordinator `runtime.properties`. +*Successfully updated retention rules for specified datasource* -`GET /druid/coordinator/v1/rules/{dataSourceName}/history?count=<n>` +<!--END_DOCUSAURUS_CODE_TABS--> -Returns last `n` entries of audit history of rules for a specified datasource. +--- + +### Sample request + +The following example sets a set of broadcast, load, and drop retention rules for the `kttm1` datasource. + +<!--DOCUSAURUS_CODE_TABS--> + +<!--cURL--> + +```shell +curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/rules/kttm1"; \ +--header 'X-Druid-Author: doc intern' \ +--header 'X-Druid-Comment: submitted via api' \ +--header 'Content-Type: application/json' \ +--data '[ + { + "type": "broadcastForever" + }, + { + "type": "loadForever", + "tieredReplicants": { + "_default_tier": 2 + }, + "useDefaultTierForNull": true + }, + { + "type": "dropByPeriod", + "period": "P1M" + } +]' +``` + +<!--HTTP--> + +```HTTP +POST /druid/coordinator/v1/rules/kttm1 HTTP/1.1 +Host: http://ROUTER_IP:ROUTER_PORT +X-Druid-Author: doc intern +X-Druid-Comment: submitted via api +Content-Type: application/json +Content-Length: 273 + +[ + { + "type": "broadcastForever" + }, + { + "type": "loadForever", + "tieredReplicants": { + "_default_tier": 1 + }, + "useDefaultTierForNull": true + }, + { + "type": "dropByPeriod", + "period": "P1M" + } +] +``` + +<!--END_DOCUSAURUS_CODE_TABS--> + +### Sample response + +A successful request returns an HTTP `200 OK` and an empty response body. + +## Update default retention rules for all datasources + +Update one or more default retention rules for all datasources. Retention rules can be submitted as an array of rule objects in the request body and overwrite any existing rules for the datasource. To remove default retention rules for all datasources, submit an empty rule array in the request body. Rules are read in the order in which they appear, see [rule structure](../operations/rule-configuration.md) for more information. Review Comment: ```suggestion Updates one or more default retention rules for all datasources. Submit retention rules as an array of objects in the request body. This request overwrites any existing rules for all datasources. To remove default retention rules for all datasources, submit an empty rule array in the request body. Rules are read in the order in which they appear; for more information, see [rule structure](../operations/rule-configuration.md). ``` ########## docs/api-reference/retention-rules-api.md: ########## @@ -23,47 +23,487 @@ sidebar_label: Retention rules ~ under the License. --> -This document describes the API endpoints for managing retention rules in Apache Druid. +This topic describes the API endpoints for managing retention rules in Apache Druid. You can configure retention rules in the Druid web console or API. -## Retention rules +Druid uses retention rules to determine what data is retained in the cluster. Druid supports load, drop, and broadcast rules. See [using rules to drop and retain data](../operations/rule-configuration.md) for more information. -Note that all _interval_ URL parameters are ISO 8601 strings delimited by a `_` instead of a `/` as in `2016-06-27_2016-06-28`. +In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments. -`GET /druid/coordinator/v1/rules` +## Update retention rules for a datasource -Returns all rules as JSON objects for all datasources in the cluster including the default datasource. +Update one or more retention rules for a datasource. Retention rules can be submitted as an array of rule objects in the request body and overwrite any existing rules for the datasource. Rules are read in the order in which they appear, see [rule structure](../operations/rule-configuration.md) for more information. -`GET /druid/coordinator/v1/rules/{dataSourceName}` +Note that this endpoint returns an `HTTP 200` even if the datasource does not exist. -Returns all rules for a specified datasource. +### URL -`GET /druid/coordinator/v1/rules/{dataSourceName}?full` +<code class="postAPI">POST</code> <code>/druid/coordinator/v1/rules/:datasource</code> -Returns all rules for a specified datasource and includes default datasource. +### Header parameters -`GET /druid/coordinator/v1/rules/history?interval=<interval>` +The endpoint supports a set of optional header parameters to populate the `author` and `comment` fields in the `auditInfo` property for audit history. -Returns audit history of rules for all datasources. Default value of interval can be specified by setting `druid.audit.manager.auditHistoryMillis` (1 week if not configured) in Coordinator `runtime.properties`. +* `X-Druid-Author` (optional) + * Type: String + * A string representing the author making the configuration change. +* `X-Druid-Comment` (optional) + * Type: String + * A string describing the update. -`GET /druid/coordinator/v1/rules/history?count=<n>` +### Responses -Returns last `n` entries of audit history of rules for all datasources. +<!--DOCUSAURUS_CODE_TABS--> -`GET /druid/coordinator/v1/rules/{dataSourceName}/history?interval=<interval>` +<!--200 SUCCESS--> -Returns audit history of rules for a specified datasource. Default value of interval can be specified by setting `druid.audit.manager.auditHistoryMillis` (1 week if not configured) in Coordinator `runtime.properties`. +*Successfully updated retention rules for specified datasource* -`GET /druid/coordinator/v1/rules/{dataSourceName}/history?count=<n>` +<!--END_DOCUSAURUS_CODE_TABS--> -Returns last `n` entries of audit history of rules for a specified datasource. +--- + +### Sample request + +The following example sets a set of broadcast, load, and drop retention rules for the `kttm1` datasource. + +<!--DOCUSAURUS_CODE_TABS--> + +<!--cURL--> + +```shell +curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/rules/kttm1"; \ +--header 'X-Druid-Author: doc intern' \ +--header 'X-Druid-Comment: submitted via api' \ +--header 'Content-Type: application/json' \ +--data '[ + { + "type": "broadcastForever" + }, + { + "type": "loadForever", + "tieredReplicants": { + "_default_tier": 2 + }, + "useDefaultTierForNull": true + }, + { + "type": "dropByPeriod", + "period": "P1M" + } +]' +``` + +<!--HTTP--> + +```HTTP +POST /druid/coordinator/v1/rules/kttm1 HTTP/1.1 +Host: http://ROUTER_IP:ROUTER_PORT +X-Druid-Author: doc intern +X-Druid-Comment: submitted via api +Content-Type: application/json +Content-Length: 273 + +[ + { + "type": "broadcastForever" + }, + { + "type": "loadForever", + "tieredReplicants": { + "_default_tier": 1 + }, + "useDefaultTierForNull": true + }, + { + "type": "dropByPeriod", + "period": "P1M" + } +] +``` + +<!--END_DOCUSAURUS_CODE_TABS--> + +### Sample response + +A successful request returns an HTTP `200 OK` and an empty response body. + +## Update default retention rules for all datasources + +Update one or more default retention rules for all datasources. Retention rules can be submitted as an array of rule objects in the request body and overwrite any existing rules for the datasource. To remove default retention rules for all datasources, submit an empty rule array in the request body. Rules are read in the order in which they appear, see [rule structure](../operations/rule-configuration.md) for more information. + +### URL + +<code class="postAPI">POST</code> <code>/druid/coordinator/v1/rules/_default</code> + +### Header parameters + +The endpoint supports a set of optional header parameters to populate the `author` and `comment` fields in the `auditInfo` property for audit history. + +* `X-Druid-Author` (optional) + * Type: String + * A string representing the author making the configuration change. +* `X-Druid-Comment` (optional) + * Type: String + * A string describing the update. + +### Responses + +<!--DOCUSAURUS_CODE_TABS--> + +<!--200 SUCCESS--> + +*Successfully updated default retention rules* + +<!--500 SERVER ERROR--> + +*Error with request body* + +<!--END_DOCUSAURUS_CODE_TABS--> + +--- + +### Sample request + +The following example updates the default retention rule for all datasources with a `loadByInterval` rule. + +<!--DOCUSAURUS_CODE_TABS--> + +<!--cURL--> + +```shell +curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/rules/_default"; \ +--header 'Content-Type: application/json' \ +--data '[ + { + "type": "loadByInterval", + "tieredReplicants": {}, + "useDefaultTierForNull": false, + "interval": "2010-01-01/2020-01-01" + } +]' +``` + +<!--HTTP--> + +```HTTP +POST /druid/coordinator/v1/rules/_default HTTP/1.1 +Host: http://ROUTER_IP:ROUTER_PORT +Content-Type: application/json +Content-Length: 205 + +[ + { + "type": "loadByInterval", + "tieredReplicants": {}, + "useDefaultTierForNull": false, + "interval": "2010-01-01/2020-01-01" + } +] +``` + +<!--END_DOCUSAURUS_CODE_TABS--> + +### Sample response + +A successful request returns an HTTP `200 OK` and an empty response body. + +## Get an array of all retention rules + +Retrieves all current retention rules in the cluster including the default retention rule. Returns an array of objects for each datasource and their associated retention rule. Review Comment: ```suggestion Retrieves all current retention rules in the cluster including the default retention rule. Returns an array of objects for each datasource and their associated retention rules. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
