This is an automated email from the ASF dual-hosted git repository.
lakshsingla pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new 3c72ec8413f docs: Migration guide for subquery limit (#16519)
3c72ec8413f is described below
commit 3c72ec8413fc066dbb56e9f02d6fbc6e3b28bbe1
Author: Jill Osborne <[email protected]>
AuthorDate: Fri May 31 04:56:07 2024 +0100
docs: Migration guide for subquery limit (#16519)
Adds a migration guide for Druid 30 to help users understand the new
byte-based subquery limit property maxSubqueryBytes
---
docs/release-info/migr-subquery-limit.md | 60 ++++++++++++++++++++++++++++++++
website/sidebars.json | 3 +-
2 files changed, 62 insertions(+), 1 deletion(-)
diff --git a/docs/release-info/migr-subquery-limit.md
b/docs/release-info/migr-subquery-limit.md
new file mode 100644
index 00000000000..853be50294d
--- /dev/null
+++ b/docs/release-info/migr-subquery-limit.md
@@ -0,0 +1,60 @@
+---
+id: migr-subquery-limit
+title: "Migration guide: Subquery limit"
+sidebar_label: Subquery limit
+---
+
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one
+ ~ or more contributor license agreements. See the NOTICE file
+ ~ distributed with this work for additional information
+ ~ regarding copyright ownership. The ASF licenses this file
+ ~ to you under the Apache License, Version 2.0 (the
+ ~ "License"); you may not use this file except in compliance
+ ~ with the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing,
+ ~ software distributed under the License is distributed on an
+ ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ ~ KIND, either express or implied. See the License for the
+ ~ specific language governing permissions and limitations
+ ~ under the License.
+-->
+
+Druid now allows you to set a byte-based limit on subquery size, to prevent
brokers from running out of memory when handling large subqueries.
+Druid uses subqueries as joins as well as in common table expressions, such as
WITH.
+
+The byte-based subquery limit overrides Druid's row-based subquery limit.
+
+:::info
+We recommend that you move towards using byte-based limits starting in Druid
30.0.
+:::
+
+For queries that generate a large number of rows (5 million or more), we
recommend that you don't use `maxSubqueryBytes` from the outset.
+You can increase `maxSubqueryRows` and then configure the byte-based limit if
you find that Druid needs it to process the query.
+
+## Row-based subquery limit
+
+Druid uses the `maxSubqueryRows` property to limit the number of rows Druid
returns in a subquery.
+Because this is a row-based limit, it doesn't restrict the overall size of the
returned data.
+
+The `maxSubqueryRows` property is set to 100,000 by default.
+
+## Enable a byte-based subquery limit
+
+Set the optional property `maxSubqueryBytes` to set a maximum number of
returned bytes.
+This property takes precedence over `maxSubqueryRows`.
+
+## Usage considerations
+
+You can set both `maxSubqueryRows` and `maxSubqueryBytes` at cluster level and
override them in individual queries.
+See [Overriding default query context
values](../configuration#overriding-default-query-context-values) for more
information.
+
+## Learn more
+
+See the following topics for more information:
+
+- [Query context](../querying/query-context.md) for information on setting
query context parameters.
+- [Broker configuration
reference](../configuration#guardrails-for-materialization-of-subqueries) for
more information on `maxSubqueryRows` and `maxSubqueryBytes`.
diff --git a/website/sidebars.json b/website/sidebars.json
index 286269953d9..5e07945ca7d 100644
--- a/website/sidebars.json
+++ b/website/sidebars.json
@@ -363,9 +363,10 @@
"id": "release-info/migration-guide"
},
"items": [
+ "release-info/migr-subquery-limit"
]
}
]},
"misc/papers-and-talks"
]
-}
\ No newline at end of file
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
