This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new c37acc1c301 [Fix](nereids) Fix session variable not take effect for
partial update in multi-statement batch (#60803)
c37acc1c301 is described below
commit c37acc1c301f2baaeeb03d83713987e3863a9502
Author: bobhan1 <[email protected]>
AuthorDate: Fri Feb 27 10:35:09 2026 +0800
[Fix](nereids) Fix session variable not take effect for partial update in
multi-statement batch (#60803)
### What problem does this PR solve?
Problem Summary:
When multiple SQL statements are sent as a single COM_QUERY (e.g., via
JDBC `allowMultiQueries=true`):
`set enable_unique_key_partial_update=true; insert into t1(k,v1)
values(1,100);`
The multi-statement execution flow is:
**Before fix (bug):**
> 1. **Parse ALL statements together (before any execution)**
> - parse `SET enable_unique_key_partial_update=true` → SetVarStmt
> - parse `INSERT INTO t1(k,v1) VALUES(1,100)` → InsertIntoTableCommand
> - `visitInsertTable()` reads session var → `isPartialUpdate=false` ❌
(stale!)
> 2. **Execute statements one by one**
> - execute SetVarStmt → session var set to `true` ✅
> - execute InsertIntoTableCommand
> - `normalizePlan()` still sees `isPartialUpdate=false` from parse
phase ❌
> - partial update NOT applied, unspecified columns overwritten with
defaults
**After fix:**
> 2. **Execute statements one by one**
> - execute SetVarStmt → session var set to `true` ✅
> - execute InsertIntoTableCommand
> - `normalizePlan()` **re-reads session var** → `isPartialUpdate=true`
✅
> - partial update correctly applied, unspecified columns preserved
All statements are parsed together by `NereidsParser.parseSQL()` before
any are executed. In `LogicalPlanBuilder.visitInsertTable()`,
`isEnableUniqueKeyPartialUpdate` and `getPartialUpdateNewRowPolicy` are
read from the session variable during **parse phase**, but the preceding
SET statement hasn't executed yet, so the INSERT reads stale values.
This causes partial update to silently not take effect, overwriting
unspecified columns with default values instead of preserving them.
### Root Cause
`visitInsertTable()` →
`UnboundTableSinkCreator.createUnboundTableSinkMaybeOverwrite()` bakes
session variable values into `UnboundTableSink` at parse time. All
downstream code (`normalizePlan`, `BindSink`, `OlapInsertExecutor`,
`PhysicalPlanTranslator`) reads from the sink object, not from session
variable. So the stale value propagates through the entire pipeline.
### Fix
Re-read both `enable_unique_key_partial_update` and
`partial_update_new_row_policy` session variables from `ConnectContext`
in `InsertUtils.normalizePlanWithoutLock()`. This is the earliest point
in the **execution phase** (after SET has executed), and it's upstream
of all consumers.
Changes:
1. **UnboundTableSink.java** — remove `final` from
`partialUpdateNewKeyPolicy` field and add
`setPartialUpdateNewKeyPolicy()` setter.
2. **InsertUtils.java** — re-read both session variables from
`ConnectContext` and set them on the `UnboundTableSink` before the
existing partial update validation logic.
3. **test_partial_update_multi_stmt.out** — update expected output to
reflect that multi-statement batch partial update now works correctly
(preserves original values instead of overwriting with defaults).
## Further comment
All downstream reads come from the sink object, not from session
variable directly. Since we fix the sink's values in
`normalizePlanWithoutLock()`, all downstream code gets the correct
values without any additional changes:
| Phase | File | Reads from |
|-------|------|-----------|
| Normalize | `InsertUtils.java` | `UnboundTableSink` (fixed here) |
| Bind | `BindSink.java` | `UnboundTableSink` |
| Physical | `LogicalOlapTableSinkToPhysical.java` |
`LogicalOlapTableSink` |
| Translate | `PhysicalPlanTranslator.java` | `PhysicalOlapTableSink` |
| Execute | `OlapInsertExecutor.java` | `PhysicalOlapTableSink` |
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [x] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [x] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
---
.../doris/nereids/analyzer/UnboundTableSink.java | 6 +-
.../trees/plans/commands/insert/InsertUtils.java | 13 ++
.../test_partial_update_multi_stmt.out | 25 +++
.../test_partial_update_multi_stmt.groovy | 168 +++++++++++++++++++++
4 files changed, 211 insertions(+), 1 deletion(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/analyzer/UnboundTableSink.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/analyzer/UnboundTableSink.java
index b9290c796c2..ed04ad57372 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/analyzer/UnboundTableSink.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/analyzer/UnboundTableSink.java
@@ -50,7 +50,7 @@ public class UnboundTableSink<CHILD_TYPE extends Plan>
extends UnboundLogicalSin
private final boolean temporaryPartition;
private final List<String> partitions;
private boolean isPartialUpdate;
- private final TPartialUpdateNewRowPolicy partialUpdateNewKeyPolicy;
+ private TPartialUpdateNewRowPolicy partialUpdateNewKeyPolicy;
private final DMLCommandType dmlCommandType;
private final boolean autoDetectPartition;
@@ -126,6 +126,10 @@ public class UnboundTableSink<CHILD_TYPE extends Plan>
extends UnboundLogicalSin
this.isPartialUpdate = isPartialUpdate;
}
+ public void setPartialUpdateNewKeyPolicy(TPartialUpdateNewRowPolicy
policy) {
+ this.partialUpdateNewKeyPolicy = policy;
+ }
+
@Override
public Plan withChildren(List<Plan> children) {
Preconditions.checkArgument(children.size() == 1,
"UnboundOlapTableSink only accepts one child");
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/InsertUtils.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/InsertUtils.java
index 16da19be4ae..b5561f0d77e 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/InsertUtils.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/InsertUtils.java
@@ -295,6 +295,19 @@ public class InsertUtils {
// For JDBC External Table, we always allow certain columns to be
missing during insertion
// Specific check for non-nullable columns only if insertion is
direct VALUES or SELECT constants
}
+ // Re-read partial update settings from session variable to handle
multi-statement
+ // batches where SET and INSERT are parsed together before execution.
+ // Only apply to original INSERT statements, not DELETE/UPDATE
converted to INSERT.
+ if (unboundLogicalSink instanceof UnboundTableSink
+ && unboundLogicalSink.getDMLCommandType() ==
DMLCommandType.INSERT) {
+ ConnectContext ctx = ConnectContext.get();
+ if (ctx != null) {
+ ((UnboundTableSink<? extends Plan>) unboundLogicalSink)
+
.setPartialUpdate(ctx.getSessionVariable().isEnableUniqueKeyPartialUpdate());
+ ((UnboundTableSink<? extends Plan>) unboundLogicalSink)
+
.setPartialUpdateNewKeyPolicy(ctx.getSessionVariable().getPartialUpdateNewRowPolicy());
+ }
+ }
if (table instanceof OlapTable && ((OlapTable) table).getKeysType() ==
KeysType.UNIQUE_KEYS) {
if (unboundLogicalSink instanceof UnboundTableSink
&& ((UnboundTableSink<? extends Plan>)
unboundLogicalSink).isPartialUpdate()) {
diff --git
a/regression-test/data/unique_with_mow_p0/partial_update/test_partial_update_multi_stmt.out
b/regression-test/data/unique_with_mow_p0/partial_update/test_partial_update_multi_stmt.out
new file mode 100644
index 00000000000..546b0cf2c4a
--- /dev/null
+++
b/regression-test/data/unique_with_mow_p0/partial_update/test_partial_update_multi_stmt.out
@@ -0,0 +1,25 @@
+-- This file is automatically generated. You should know what you did if you
want to edit this
+-- !base1 --
+1 1 1 1
+2 2 2 2
+
+-- !base2 --
+1 1 1 1
+2 2 2 2
+
+-- !multi_stmt_values --
+1 100 1 1
+2 200 2 2
+
+-- !multi_stmt_select --
+1 501 1 1
+2 502 2 2
+
+-- !separate_stmt_values --
+1 100 1 1
+2 200 2 2
+
+-- !separate_stmt_select --
+1 501 1 1
+2 502 2 2
+
diff --git
a/regression-test/suites/unique_with_mow_p0/partial_update/test_partial_update_multi_stmt.groovy
b/regression-test/suites/unique_with_mow_p0/partial_update/test_partial_update_multi_stmt.groovy
new file mode 100644
index 00000000000..7dbc33b5c43
--- /dev/null
+++
b/regression-test/suites/unique_with_mow_p0/partial_update/test_partial_update_multi_stmt.groovy
@@ -0,0 +1,168 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+import java.sql.Connection
+import java.sql.DriverManager
+import java.sql.Statement
+
+// Test that `set enable_unique_key_partial_update=true; insert into ...`
works correctly
+// when sent as a single multi-statement query (single COM_QUERY).
+// This is a regression test for the issue where session variables set via SET
in a
+// multi-statement batch might not be visible during the parse phase of
subsequent
+// statements in the same batch, because all statements are parsed together
before
+// any of them are executed.
+suite("test_partial_update_multi_stmt", "p0") {
+
+ String db = context.config.getDbNameByFile(context.file)
+ sql "select 1;" // to create database
+
+ String jdbcUrl = context.config.jdbcUrl
+ String urlWithoutSchema = jdbcUrl.substring(jdbcUrl.indexOf("://") + 3)
+ def sql_ip = urlWithoutSchema.substring(0, urlWithoutSchema.indexOf(":"))
+ def sql_port
+ if (urlWithoutSchema.indexOf("/") >= 0) {
+ sql_port = urlWithoutSchema.substring(urlWithoutSchema.indexOf(":") +
1, urlWithoutSchema.indexOf("/"))
+ } else {
+ sql_port = urlWithoutSchema.substring(urlWithoutSchema.indexOf(":") +
1)
+ }
+
+ def tableName1 = "test_partial_update_multi_stmt1"
+ def tableName2 = "test_partial_update_multi_stmt2"
+ def tableName3 = "test_partial_update_multi_stmt3"
+ def tableName4 = "test_partial_update_multi_stmt4"
+
+ def createTable = { tblName ->
+ sql """ DROP TABLE IF EXISTS ${tblName} """
+ sql """
+ CREATE TABLE ${tblName} (
+ `k` int(11) NOT NULL COMMENT "key",
+ `v1` int(11) NULL DEFAULT "10" COMMENT "value1",
+ `v2` int(11) NULL DEFAULT "20" COMMENT "value2",
+ `v3` int(11) NULL DEFAULT "30" COMMENT "value3"
+ ) UNIQUE KEY(`k`)
+ DISTRIBUTED BY HASH(`k`) BUCKETS 1
+ PROPERTIES(
+ "replication_num" = "1",
+ "enable_unique_key_merge_on_write" = "true"
+ ); """
+ sql "insert into ${tblName} values(1, 1, 1, 1), (2, 2, 2, 2);"
+ sql "sync;"
+ }
+
+ // ============================================================
+ // Setup: create all tables with initial data (1,1,1,1),(2,2,2,2)
+ // ============================================================
+ connect(context.config.jdbcUser, context.config.jdbcPassword,
context.config.jdbcUrl) {
+ sql "use ${db};"
+ sql "sync;"
+ createTable(tableName1)
+ createTable(tableName2)
+ createTable(tableName3)
+ createTable(tableName4)
+ qt_base1 "select * from ${tableName1} order by k;"
+ qt_base2 "select * from ${tableName2} order by k;"
+ }
+
+ // ============================================================
+ // Test 1 & 2: multi-statement with allowMultiQueries=true
+ // SET + INSERT sent as a single COM_QUERY
+ // ============================================================
+ def multiStmtUrl =
"jdbc:mysql://${sql_ip}:${sql_port}/${db}?useLocalSessionState=false&allowMultiQueries=true"
+ logger.info("multi-statement JDBC URL: ${multiStmtUrl}")
+
+ try (Connection conn = DriverManager.getConnection(multiStmtUrl,
+ context.config.jdbcUser, context.config.jdbcPassword)) {
+ Statement stmt = conn.createStatement()
+
+ // Test 1: SET + INSERT VALUES as single COM_QUERY
+ logger.info("Test 1: allowMultiQueries=true, SET + INSERT VALUES")
+ stmt.execute(
+ "set enable_unique_key_partial_update=true;" +
+ "set enable_insert_strict=false;" +
+ "insert into ${tableName1}(k, v1) values(1, 100), (2, 200);"
+ )
+ stmt.execute("set enable_unique_key_partial_update=false;")
+
+ // Test 2: SET + INSERT SELECT as single COM_QUERY
+ logger.info("Test 2: allowMultiQueries=true, SET + INSERT SELECT")
+ stmt.execute(
+ "set enable_unique_key_partial_update=true;" +
+ "set enable_insert_strict=false;" +
+ "insert into ${tableName2}(k, v1) select k, v1 + 500 from
${tableName2};"
+ )
+ stmt.execute("set enable_unique_key_partial_update=false;")
+
+ stmt.close()
+ }
+
+ // ============================================================
+ // Test 3 & 4: without allowMultiQueries (statements sent separately)
+ // SET and INSERT are separate COM_QUERY packets
+ // ============================================================
+ def singleStmtUrl =
"jdbc:mysql://${sql_ip}:${sql_port}/${db}?useLocalSessionState=false"
+ logger.info("single-statement JDBC URL: ${singleStmtUrl}")
+
+ try (Connection conn = DriverManager.getConnection(singleStmtUrl,
+ context.config.jdbcUser, context.config.jdbcPassword)) {
+ Statement stmt = conn.createStatement()
+
+ // Test 3: SET then INSERT VALUES as separate COM_QUERY
+ logger.info("Test 3: separate statements, SET + INSERT VALUES")
+ stmt.execute("set enable_unique_key_partial_update=true;")
+ stmt.execute("set enable_insert_strict=false;")
+ stmt.execute("insert into ${tableName3}(k, v1) values(1, 100), (2,
200);")
+ stmt.execute("set enable_unique_key_partial_update=false;")
+
+ // Test 4: SET then INSERT SELECT as separate COM_QUERY
+ logger.info("Test 4: separate statements, SET + INSERT SELECT")
+ stmt.execute("set enable_unique_key_partial_update=true;")
+ stmt.execute("set enable_insert_strict=false;")
+ stmt.execute("insert into ${tableName4}(k, v1) select k, v1 + 500 from
${tableName4};")
+ stmt.execute("set enable_unique_key_partial_update=false;")
+
+ stmt.close()
+ }
+
+ // ============================================================
+ // Verify all results
+ // ============================================================
+ connect(context.config.jdbcUser, context.config.jdbcPassword,
context.config.jdbcUrl) {
+ sql "use ${db};"
+ sql "sync;"
+
+ // Test 1: allowMultiQueries + INSERT VALUES
+ // Expected (partial update): v2,v3 keep old values (1,1) and (2,2)
+ qt_multi_stmt_values "select * from ${tableName1} order by k;"
+
+ // Test 2: allowMultiQueries + INSERT SELECT
+ // Expected (partial update): v2,v3 keep old values
+ qt_multi_stmt_select "select * from ${tableName2} order by k;"
+
+ // Test 3: separate statements + INSERT VALUES
+ // Expected (partial update): v2,v3 keep old values (1,1) and (2,2)
+ qt_separate_stmt_values "select * from ${tableName3} order by k;"
+
+ // Test 4: separate statements + INSERT SELECT
+ // Expected (partial update): v2,v3 keep old values
+ qt_separate_stmt_select "select * from ${tableName4} order by k;"
+
+ sql """ DROP TABLE IF EXISTS ${tableName1} """
+ sql """ DROP TABLE IF EXISTS ${tableName2} """
+ sql """ DROP TABLE IF EXISTS ${tableName3} """
+ sql """ DROP TABLE IF EXISTS ${tableName4} """
+ }
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
