This is an automated email from the ASF dual-hosted git repository. vogievetsky pushed a commit to branch doc_polish_24v2 in repository https://gitbox.apache.org/repos/asf/druid-website-src.git
commit 83336a403ce7088f474be892136ebd2b83f5746b Author: Vadim Ogievetsky <[email protected]> AuthorDate: Sat Sep 17 14:13:30 2022 -0700 doc update --- docs/24.0.0/multi-stage-query/api.html | 25 ++++++++++++++----------- docs/24.0.0/multi-stage-query/concepts.html | 11 ++++++----- docs/24.0.0/multi-stage-query/index.html | 9 +++++---- docs/latest/multi-stage-query/api.html | 25 ++++++++++++++----------- docs/latest/multi-stage-query/concepts.html | 11 ++++++----- docs/latest/multi-stage-query/index.html | 9 +++++---- 6 files changed, 50 insertions(+), 40 deletions(-) diff --git a/docs/24.0.0/multi-stage-query/api.html b/docs/24.0.0/multi-stage-query/api.html index eb96c54a..f2c99cdb 100644 --- a/docs/24.0.0/multi-stage-query/api.html +++ b/docs/24.0.0/multi-stage-query/api.html @@ -93,13 +93,13 @@ interface.</p> <h2><a class="anchor" aria-hidden="true" id="submit-a-query"></a><a href="#submit-a-query" aria-hidden="true" class="hash-link"><svg class="hash-link-icon" aria-hidden="true" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0- [...] <p>You submit queries to the MSQ task engine using the <code>POST /druid/v2/sql/task/</code> endpoint.</p> <h4><a class="anchor" aria-hidden="true" id="request"></a><a href="#request" aria-hidden="true" class="hash-link"><svg class="hash-link-icon" aria-hidden="true" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1- [...] -<p>Currently, the MSQ task engine ignores the provided values of <code>resultFormat</code>, <code>header</code>, -<code>typesHeader</code>, and <code>sqlTypesHeader</code>. SQL SELECT queries write out their results into the task report (in the <code>multiStageQuery.payload.results.results</code> key) formatted as if <code>resultFormat</code> is an <code>array</code>.</p> -<p>For task queries similar to the <a href="/docs/24.0.0/multi-stage-query/examples.html">example queries</a>, you need to escape characters such as quotation marks (") if you use something like <code>curl</code>. -You don't need to escape characters if you use a method that can parse JSON seamlessly, such as Python. -The Python example in this topic escapes quotation marks although it's not required.</p> -<p>The following example is the same query that you submit when you complete <a href="/docs/24.0.0/tutorials/tutorial-msq-convert-spec.html">Convert a JSON ingestion -spec</a> where you insert data into a table named <code>wikipedia</code>.</p> +<p>The SQL task endpoint accepts <a href="/docs/24.0.0/querying/sql-api.html#request-body">SQL requests in the JSON-over-HTTP form</a> using the +<code>query</code>, <code>context</code>, and <code>parameters</code> fields, but ignoring the <code>resultFormat</code>, <code>header</code>, <code>typesHeader</code>, and +<code>sqlTypesHeader</code> fields.</p> +<p>This endpoint accepts <a href="/docs/24.0.0/multi-stage-query/reference.html#insert">INSERT</a> and <a href="/docs/24.0.0/multi-stage-query/reference.html#replace">REPLACE</a> statements.</p> +<p>As an experimental feature, this endpoint also accepts SELECT queries. SELECT query results are collected from workers +by the controller, and written into the <a href="#get-the-report-for-a-query-task">task report</a> as an array of arrays. The +behavior and result format of plain SELECT queries (without INSERT or REPLACE) is subject to change.</p> <div class="tabs"><div class="nav-tabs"><div id="tab-group-1-tab-2" class="nav-link active" data-group="group_1" data-tab="tab-group-1-content-2">HTTP</div><div id="tab-group-1-tab-3" class="nav-link" data-group="group_1" data-tab="tab-group-1-content-3">curl</div><div id="tab-group-1-tab-4" class="nav-link" data-group="group_1" data-tab="tab-group-1-content-4">Python</div></div><div class="tab-content"><div id="tab-group-1-content-2" class="tab-pane active" data-group="group_1" tabindex [...] <pre><code class="hljs css language-json">{<br /> <span class="hljs-attr">"query"</span>: <span class="hljs-string">"INSERT INTO wikipedia\nSELECT\n TIME_PARSE(\"timestamp\") AS __time,\n *\nFROM TABLE(\n EXTERN(\n '{\"type\": \"http\", \"uris\": [\"https://druid.apache.org/data/wikipedia.json.gz\"]}',\n '{\"type\": \"json\"}',\n '[{\"name\": \"added\", \"type\": \"long\"}, {\"name\": \"channel\", \"type\": \"string\"}, {\"name\": \"cityName\", \"type\": \"string\"}, {\"name [...] </span></div></div><div id="tab-group-1-content-3" class="tab-pane" data-group="group_1" tabindex="-1"><div><span><pre><code class="hljs css language-bash"><span class="hljs-comment"># Make sure you replace `username`, `password`, `your-instance`, and `port` with the values for your deployment.</span><br />curl --location --request POST <span class="hljs-string">'https://<username>:<password>@<your-instance>:<port>/druid/v2/sql/task/'</span> \<br /> --header <spa [...] @@ -155,15 +155,18 @@ spec</a> where you insert data into a table named <code>wikipedia</code>.</p> <p>A report provides detailed information about a query task, including things like the stages, warnings, and errors.</p> <p>Keep the following in mind when using the task API to view reports:</p> <ul> -<li>For SELECT queries, the report includes the results. At this time, if you want to view results for SELECT queries, you need to retrieve them as a generic map from the report and extract the results.</li> -<li>The task report stores query details for controller tasks.</li> -<li>If you encounter <code>500 Server Error</code> or <code>404 Not Found</code> errors, the task may be in the process of starting up or shutting down.</li> +<li>The task report for an entire job is associated with the <code>query_controller</code> task. The <code>query_worker</code> tasks do not have +their own reports; their information is incorporated into the controller report.</li> +<li>The task report API may report <code>404 Not Found</code> temporarily while the task is in the process of starting up.</li> +<li>As an experimental feature, the SQL task engine supports running SELECT queries. SELECT query results are written into +the <code>multiStageQuery.payload.results.results</code> task report key as an array of arrays. The behavior and result format of plain +SELECT queries (without INSERT or REPLACE) is subject to change.</li> </ul> <p>For an explanation of the fields in a report, see <a href="#report-response-fields">Report response fields</a>.</p> <h4><a class="anchor" aria-hidden="true" id="request-2"></a><a href="#request-2" aria-hidden="true" class="hash-link"><svg class="hash-link-icon" aria-hidden="true" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.6 [...] <div class="tabs"><div class="nav-tabs"><div id="tab-group-9-tab-10" class="nav-link active" data-group="group_9" data-tab="tab-group-9-content-10">HTTP</div><div id="tab-group-9-tab-11" class="nav-link" data-group="group_9" data-tab="tab-group-9-content-11">curl</div><div id="tab-group-9-tab-12" class="nav-link" data-group="group_9" data-tab="tab-group-9-content-12">Python</div></div><div class="tab-content"><div id="tab-group-9-content-10" class="tab-pane active" data-group="group_9" t [...] </span></div></div><div id="tab-group-9-content-11" class="tab-pane" data-group="group_9" tabindex="-1"><div><span><pre><code class="hljs css language-bash"><span class="hljs-comment"># Make sure you replace `username`, `password`, `your-instance`, `port`, and `taskId` with the values for your deployment.</span><br />curl --location --request GET <span class="hljs-string">'https://<username>:<password>@<hostname>:<port>/druid/indexer/v1/task/<taskId>/reports [...] -</span></div></div><div id="tab-group-9-content-12" class="tab-pane" data-group="group_9" tabindex="-1"><div><span><pre><code class="hljs css language-python"><span class="hljs-keyword">import</span> requests<br /><br /><span class="hljs-comment"># Make sure you replace `username`, `password`, `your-instance`, `port`, and `taskId` with the values for your deployment.</span><br />url = <span class="hljs-string">"https://<username>:<password>@<hostname>:<port>/druid [...] +</span></div></div><div id="tab-group-9-content-12" class="tab-pane" data-group="group_9" tabindex="-1"><div><span><pre><code class="hljs css language-python"><span class="hljs-keyword">import</span> requests<br /><br /><span class="hljs-comment"># Make sure you replace `username`, `password`, `your-instance`, `port`, and `taskId` with the values for your deployment.</span><br />url = <span class="hljs-string">"https://<username>:<password>@<hostname>:<port>/druid [...] </span></div></div></div></div> <h4><a class="anchor" aria-hidden="true" id="response-2"></a><a href="#response-2" aria-hidden="true" class="hash-link"><svg class="hash-link-icon" aria-hidden="true" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1 [...] <p>The response shows an example report for a query.</p> diff --git a/docs/24.0.0/multi-stage-query/concepts.html b/docs/24.0.0/multi-stage-query/concepts.html index a1145efc..2ddcb85e 100644 --- a/docs/24.0.0/multi-stage-query/concepts.html +++ b/docs/24.0.0/multi-stage-query/concepts.html @@ -83,13 +83,14 @@ extension, new in Druid 24.0. Refer to the <a href="/docs/24.0.0/ingestion/index ingestion method is right for you.</p> </blockquote> <h2><a class="anchor" aria-hidden="true" id="sql-task-engine"></a><a href="#sql-task-engine" aria-hidden="true" class="hash-link"><svg class="hash-link-icon" aria-hidden="true" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 [...] -<p>The <code>druid-multi-stage-query</code> extension adds a multi-stage query (MSQ) task engine that executes SQL SELECT, -<a href="/docs/24.0.0/multi-stage-query/reference.html#insert">INSERT</a>, and <a href="/docs/24.0.0/multi-stage-query/reference.html#replace">REPLACE</a> statements as batch tasks in the indexing service, -which execute on <a href="/docs/24.0.0/design/architecture.html#druid-services">Middle Managers</a>. INSERT and REPLACE tasks publish +<p>The <code>druid-multi-stage-query</code> extension adds a multi-stage query (MSQ) task engine that executes SQL statements as batch +tasks in the indexing service, which execute on <a href="/docs/24.0.0/design/architecture.html#druid-services">Middle Managers</a>. +<a href="/docs/24.0.0/multi-stage-query/reference.html#insert">INSERT</a> and <a href="/docs/24.0.0/multi-stage-query/reference.html#replace">REPLACE</a> tasks publish <a href="/docs/24.0.0/design/architecture.html#datasources-and-segments">segments</a> just like <a href="/docs/24.0.0/ingestion/index.html#batch">all other forms of batch ingestion</a>. Each query occupies at least two task slots while running: one controller task, -and at least one worker task.</p> -<p>You can execute queries using the MSQ task engine through the <strong>Query</strong> view in the <a href="/docs/24.0.0/operations/web-console.html">web +and at least one worker task. As an experimental feature, the MSQ task engine also supports running SELECT queries as +batch tasks. The behavior and result format of plain SELECT (without INSERT or REPLACE) is subject to change.</p> +<p>You can execute SQL statements using the MSQ task engine through the <strong>Query</strong> view in the <a href="/docs/24.0.0/operations/web-console.html">web console</a> or through the <a href="/docs/24.0.0/multi-stage-query/api.html"><code>/druid/v2/sql/task</code> API</a>.</p> <p>For more details on how SQL queries are executed using the MSQ task engine, see <a href="#multi-stage-query-tasks">multi-stage query tasks</a>.</p> diff --git a/docs/24.0.0/multi-stage-query/index.html b/docs/24.0.0/multi-stage-query/index.html index 9a94db7e..591abbed 100644 --- a/docs/24.0.0/multi-stage-query/index.html +++ b/docs/24.0.0/multi-stage-query/index.html @@ -84,10 +84,11 @@ ingestion method is right for you.</p> </blockquote> <p>Apache Druid supports SQL-based ingestion using the bundled <a href="#load-the-extension"><code>druid-multi-stage-query</code> extension</a>. This extension adds a <a href="/docs/24.0.0/multi-stage-query/concepts.html#sql-task-engine">multi-stage query task engine for SQL</a> that allows running SQL -<a href="/docs/24.0.0/multi-stage-query/concepts.html#insert">INSERT</a> and <a href="/docs/24.0.0/multi-stage-query/concepts.html#replace">REPLACE</a> statements as batch tasks.</p> -<p>Nearly all SELECT capabilities are available for <code>INSERT ... SELECT</code> and <code>REPLACE ... SELECT</code> queries, with certain -exceptions listed on the <a href="/docs/24.0.0/multi-stage-query/known-issues.html#select">Known issues</a> page. This allows great flexibility to apply -transformations, filters, JOINs, aggregations, and so on while ingesting data. This also allows in-database +<a href="/docs/24.0.0/multi-stage-query/concepts.html#insert">INSERT</a> and <a href="/docs/24.0.0/multi-stage-query/concepts.html#replace">REPLACE</a> statements as batch tasks. As an experimental feature, +the task engine also supports running SELECT queries as batch tasks.</p> +<p>Nearly all SELECT capabilities are available in the SQL task engine, with certain exceptions listed on the <a href="/docs/24.0.0/multi-stage-query/known-issues.html#select">Known +issues</a> page. This allows great flexibility to apply transformations, filters, JOINs, +aggregations, and so on as part of <code>INSERT ... SELECT</code> and <code>REPLACE ... SELECT</code> statements. This also allows in-database transformation: creating new tables based on queries of other tables.</p> <h2><a class="anchor" aria-hidden="true" id="vocabulary"></a><a href="#vocabulary" aria-hidden="true" class="hash-link"><svg class="hash-link-icon" aria-hidden="true" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1 [...] <ul> diff --git a/docs/latest/multi-stage-query/api.html b/docs/latest/multi-stage-query/api.html index 9b468d77..54b95cfb 100644 --- a/docs/latest/multi-stage-query/api.html +++ b/docs/latest/multi-stage-query/api.html @@ -93,13 +93,13 @@ interface.</p> <h2><a class="anchor" aria-hidden="true" id="submit-a-query"></a><a href="#submit-a-query" aria-hidden="true" class="hash-link"><svg class="hash-link-icon" aria-hidden="true" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0- [...] <p>You submit queries to the MSQ task engine using the <code>POST /druid/v2/sql/task/</code> endpoint.</p> <h4><a class="anchor" aria-hidden="true" id="request"></a><a href="#request" aria-hidden="true" class="hash-link"><svg class="hash-link-icon" aria-hidden="true" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1- [...] -<p>Currently, the MSQ task engine ignores the provided values of <code>resultFormat</code>, <code>header</code>, -<code>typesHeader</code>, and <code>sqlTypesHeader</code>. SQL SELECT queries write out their results into the task report (in the <code>multiStageQuery.payload.results.results</code> key) formatted as if <code>resultFormat</code> is an <code>array</code>.</p> -<p>For task queries similar to the <a href="/docs/latest/multi-stage-query/examples.html">example queries</a>, you need to escape characters such as quotation marks (") if you use something like <code>curl</code>. -You don't need to escape characters if you use a method that can parse JSON seamlessly, such as Python. -The Python example in this topic escapes quotation marks although it's not required.</p> -<p>The following example is the same query that you submit when you complete <a href="/docs/latest/tutorials/tutorial-msq-convert-spec.html">Convert a JSON ingestion -spec</a> where you insert data into a table named <code>wikipedia</code>.</p> +<p>The SQL task endpoint accepts <a href="/docs/latest/querying/sql-api.html#request-body">SQL requests in the JSON-over-HTTP form</a> using the +<code>query</code>, <code>context</code>, and <code>parameters</code> fields, but ignoring the <code>resultFormat</code>, <code>header</code>, <code>typesHeader</code>, and +<code>sqlTypesHeader</code> fields.</p> +<p>This endpoint accepts <a href="/docs/latest/multi-stage-query/reference.html#insert">INSERT</a> and <a href="/docs/latest/multi-stage-query/reference.html#replace">REPLACE</a> statements.</p> +<p>As an experimental feature, this endpoint also accepts SELECT queries. SELECT query results are collected from workers +by the controller, and written into the <a href="#get-the-report-for-a-query-task">task report</a> as an array of arrays. The +behavior and result format of plain SELECT queries (without INSERT or REPLACE) is subject to change.</p> <div class="tabs"><div class="nav-tabs"><div id="tab-group-1-tab-2" class="nav-link active" data-group="group_1" data-tab="tab-group-1-content-2">HTTP</div><div id="tab-group-1-tab-3" class="nav-link" data-group="group_1" data-tab="tab-group-1-content-3">curl</div><div id="tab-group-1-tab-4" class="nav-link" data-group="group_1" data-tab="tab-group-1-content-4">Python</div></div><div class="tab-content"><div id="tab-group-1-content-2" class="tab-pane active" data-group="group_1" tabindex [...] <pre><code class="hljs css language-json">{<br /> <span class="hljs-attr">"query"</span>: <span class="hljs-string">"INSERT INTO wikipedia\nSELECT\n TIME_PARSE(\"timestamp\") AS __time,\n *\nFROM TABLE(\n EXTERN(\n '{\"type\": \"http\", \"uris\": [\"https://druid.apache.org/data/wikipedia.json.gz\"]}',\n '{\"type\": \"json\"}',\n '[{\"name\": \"added\", \"type\": \"long\"}, {\"name\": \"channel\", \"type\": \"string\"}, {\"name\": \"cityName\", \"type\": \"string\"}, {\"name [...] </span></div></div><div id="tab-group-1-content-3" class="tab-pane" data-group="group_1" tabindex="-1"><div><span><pre><code class="hljs css language-bash"><span class="hljs-comment"># Make sure you replace `username`, `password`, `your-instance`, and `port` with the values for your deployment.</span><br />curl --location --request POST <span class="hljs-string">'https://<username>:<password>@<your-instance>:<port>/druid/v2/sql/task/'</span> \<br /> --header <spa [...] @@ -155,15 +155,18 @@ spec</a> where you insert data into a table named <code>wikipedia</code>.</p> <p>A report provides detailed information about a query task, including things like the stages, warnings, and errors.</p> <p>Keep the following in mind when using the task API to view reports:</p> <ul> -<li>For SELECT queries, the report includes the results. At this time, if you want to view results for SELECT queries, you need to retrieve them as a generic map from the report and extract the results.</li> -<li>The task report stores query details for controller tasks.</li> -<li>If you encounter <code>500 Server Error</code> or <code>404 Not Found</code> errors, the task may be in the process of starting up or shutting down.</li> +<li>The task report for an entire job is associated with the <code>query_controller</code> task. The <code>query_worker</code> tasks do not have +their own reports; their information is incorporated into the controller report.</li> +<li>The task report API may report <code>404 Not Found</code> temporarily while the task is in the process of starting up.</li> +<li>As an experimental feature, the SQL task engine supports running SELECT queries. SELECT query results are written into +the <code>multiStageQuery.payload.results.results</code> task report key as an array of arrays. The behavior and result format of plain +SELECT queries (without INSERT or REPLACE) is subject to change.</li> </ul> <p>For an explanation of the fields in a report, see <a href="#report-response-fields">Report response fields</a>.</p> <h4><a class="anchor" aria-hidden="true" id="request-2"></a><a href="#request-2" aria-hidden="true" class="hash-link"><svg class="hash-link-icon" aria-hidden="true" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.6 [...] <div class="tabs"><div class="nav-tabs"><div id="tab-group-9-tab-10" class="nav-link active" data-group="group_9" data-tab="tab-group-9-content-10">HTTP</div><div id="tab-group-9-tab-11" class="nav-link" data-group="group_9" data-tab="tab-group-9-content-11">curl</div><div id="tab-group-9-tab-12" class="nav-link" data-group="group_9" data-tab="tab-group-9-content-12">Python</div></div><div class="tab-content"><div id="tab-group-9-content-10" class="tab-pane active" data-group="group_9" t [...] </span></div></div><div id="tab-group-9-content-11" class="tab-pane" data-group="group_9" tabindex="-1"><div><span><pre><code class="hljs css language-bash"><span class="hljs-comment"># Make sure you replace `username`, `password`, `your-instance`, `port`, and `taskId` with the values for your deployment.</span><br />curl --location --request GET <span class="hljs-string">'https://<username>:<password>@<hostname>:<port>/druid/indexer/v1/task/<taskId>/reports [...] -</span></div></div><div id="tab-group-9-content-12" class="tab-pane" data-group="group_9" tabindex="-1"><div><span><pre><code class="hljs css language-python"><span class="hljs-keyword">import</span> requests<br /><br /><span class="hljs-comment"># Make sure you replace `username`, `password`, `your-instance`, `port`, and `taskId` with the values for your deployment.</span><br />url = <span class="hljs-string">"https://<username>:<password>@<hostname>:<port>/druid [...] +</span></div></div><div id="tab-group-9-content-12" class="tab-pane" data-group="group_9" tabindex="-1"><div><span><pre><code class="hljs css language-python"><span class="hljs-keyword">import</span> requests<br /><br /><span class="hljs-comment"># Make sure you replace `username`, `password`, `your-instance`, `port`, and `taskId` with the values for your deployment.</span><br />url = <span class="hljs-string">"https://<username>:<password>@<hostname>:<port>/druid [...] </span></div></div></div></div> <h4><a class="anchor" aria-hidden="true" id="response-2"></a><a href="#response-2" aria-hidden="true" class="hash-link"><svg class="hash-link-icon" aria-hidden="true" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1 [...] <p>The response shows an example report for a query.</p> diff --git a/docs/latest/multi-stage-query/concepts.html b/docs/latest/multi-stage-query/concepts.html index 21cc94c7..073306de 100644 --- a/docs/latest/multi-stage-query/concepts.html +++ b/docs/latest/multi-stage-query/concepts.html @@ -83,13 +83,14 @@ extension, new in Druid 24.0. Refer to the <a href="/docs/latest/ingestion/index ingestion method is right for you.</p> </blockquote> <h2><a class="anchor" aria-hidden="true" id="sql-task-engine"></a><a href="#sql-task-engine" aria-hidden="true" class="hash-link"><svg class="hash-link-icon" aria-hidden="true" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 [...] -<p>The <code>druid-multi-stage-query</code> extension adds a multi-stage query (MSQ) task engine that executes SQL SELECT, -<a href="/docs/latest/multi-stage-query/reference.html#insert">INSERT</a>, and <a href="/docs/latest/multi-stage-query/reference.html#replace">REPLACE</a> statements as batch tasks in the indexing service, -which execute on <a href="/docs/latest/design/architecture.html#druid-services">Middle Managers</a>. INSERT and REPLACE tasks publish +<p>The <code>druid-multi-stage-query</code> extension adds a multi-stage query (MSQ) task engine that executes SQL statements as batch +tasks in the indexing service, which execute on <a href="/docs/latest/design/architecture.html#druid-services">Middle Managers</a>. +<a href="/docs/latest/multi-stage-query/reference.html#insert">INSERT</a> and <a href="/docs/latest/multi-stage-query/reference.html#replace">REPLACE</a> tasks publish <a href="/docs/latest/design/architecture.html#datasources-and-segments">segments</a> just like <a href="/docs/latest/ingestion/index.html#batch">all other forms of batch ingestion</a>. Each query occupies at least two task slots while running: one controller task, -and at least one worker task.</p> -<p>You can execute queries using the MSQ task engine through the <strong>Query</strong> view in the <a href="/docs/latest/operations/web-console.html">web +and at least one worker task. As an experimental feature, the MSQ task engine also supports running SELECT queries as +batch tasks. The behavior and result format of plain SELECT (without INSERT or REPLACE) is subject to change.</p> +<p>You can execute SQL statements using the MSQ task engine through the <strong>Query</strong> view in the <a href="/docs/latest/operations/web-console.html">web console</a> or through the <a href="/docs/latest/multi-stage-query/api.html"><code>/druid/v2/sql/task</code> API</a>.</p> <p>For more details on how SQL queries are executed using the MSQ task engine, see <a href="#multi-stage-query-tasks">multi-stage query tasks</a>.</p> diff --git a/docs/latest/multi-stage-query/index.html b/docs/latest/multi-stage-query/index.html index f955aa2a..17f79cc3 100644 --- a/docs/latest/multi-stage-query/index.html +++ b/docs/latest/multi-stage-query/index.html @@ -84,10 +84,11 @@ ingestion method is right for you.</p> </blockquote> <p>Apache Druid supports SQL-based ingestion using the bundled <a href="#load-the-extension"><code>druid-multi-stage-query</code> extension</a>. This extension adds a <a href="/docs/latest/multi-stage-query/concepts.html#sql-task-engine">multi-stage query task engine for SQL</a> that allows running SQL -<a href="/docs/latest/multi-stage-query/concepts.html#insert">INSERT</a> and <a href="/docs/latest/multi-stage-query/concepts.html#replace">REPLACE</a> statements as batch tasks.</p> -<p>Nearly all SELECT capabilities are available for <code>INSERT ... SELECT</code> and <code>REPLACE ... SELECT</code> queries, with certain -exceptions listed on the <a href="/docs/latest/multi-stage-query/known-issues.html#select">Known issues</a> page. This allows great flexibility to apply -transformations, filters, JOINs, aggregations, and so on while ingesting data. This also allows in-database +<a href="/docs/latest/multi-stage-query/concepts.html#insert">INSERT</a> and <a href="/docs/latest/multi-stage-query/concepts.html#replace">REPLACE</a> statements as batch tasks. As an experimental feature, +the task engine also supports running SELECT queries as batch tasks.</p> +<p>Nearly all SELECT capabilities are available in the SQL task engine, with certain exceptions listed on the <a href="/docs/latest/multi-stage-query/known-issues.html#select">Known +issues</a> page. This allows great flexibility to apply transformations, filters, JOINs, +aggregations, and so on as part of <code>INSERT ... SELECT</code> and <code>REPLACE ... SELECT</code> statements. This also allows in-database transformation: creating new tables based on queries of other tables.</p> <h2><a class="anchor" aria-hidden="true" id="vocabulary"></a><a href="#vocabulary" aria-hidden="true" class="hash-link"><svg class="hash-link-icon" aria-hidden="true" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1 [...] <ul> --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
