This is an automated email from the ASF dual-hosted git repository.
Gabriel39 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new fc29f25cfaa [feature](catalog) Add MongoDB session read policies
(#70493)
fc29f25cfaa is described below
commit fc29f25cfaa15f7324d8cde22d3c96966143b254
Author: Gabriel <[email protected]>
AuthorDate: Mon Nov 16 11:20:25 2026 +0800
[feature](catalog) Add MongoDB session read policies (#70493)
### Background
The native MongoDB catalog sends read preference and read concern to the
BE in each scan descriptor. Today those values come only from catalog
properties or the connection URI, while metadata discovery uses its own
primary/default policy.
### What problem does this PR solve?
Issue Number: #70251
Related PR: #70327 (native reader and catalog read policies)
PR Type: Feature
Problem Summary:
Users cannot choose MongoDB routing and visibility for an individual
session or query without changing the shared catalog. Add
`mongodb_read_preference` and `mongodb_read_concern`, supporting `SET`
and `SET_VAR`, and pin their effective values in each scan descriptor.
Both default to `inherit`, preserving existing catalog/URI behavior.
Most of the diff adds tests and generated regression expectations.
### Details
- **Change map:** 70 production lines changed, 281 test-source lines
changed, 62 generated expectation lines, and 38 documentation lines
changed.
- **Scope:** Data-read policies for native MongoDB scans. Preference
supports `inherit`, `primary`, `primaryPreferred`, `secondary`,
`secondaryPreferred`, and `nearest`; concern supports `inherit`,
`local`, `majority`, and `available`. Values are case-insensitive.
Snapshot and linearizable reads remain unsupported.
- **Design:** Register validated, forwarded session variables and
resolve them through the existing connector session-property snapshot.
Reuse the existing scan descriptor and BE collection policy application.
Session overrides do not mutate catalog configuration, metadata clients,
or transport pool identity. URI tags, maximum staleness, and hedge
options remain effective; an incompatible primary override fails during
planning without exposing the URI.
- **Usage:** `SET mongodb_read_preference='secondaryPreferred'; SET
mongodb_read_concern='local';` or `SELECT /*+
SET_VAR(mongodb_read_preference='primary',
mongodb_read_concern='majority') */ ...`. Set either variable to
`inherit` to use its catalog/URI policy again.
- **Default behavior:** `inherit` independently resolves each policy
from URI options first, then catalog defaults. Queries without overrides
retain their existing behavior.
- **Compatibility:** No connector SPI, Thrift, BE code, or
storage-format changes. Metadata discovery and collection-identity
verification retain their existing policies. Read concern does not
establish a snapshot shared by scans.
### Testing
All checks below ran locally against MongoDB 8.0.20, with the new FE and
the existing ASAN native BE:
- Red/green: the new scan-plan tests first failed because overrides were
ignored, and the new session tests first failed because the variables
were unregistered; both passed after implementation.
- `run-fe-ut.sh --run` with `SessionVariableMongoReadPolicyTest`,
`VariableMgrExportTest`, `MongoScanPlanTest`, and
`MongoCatalogPropertiesTest`: passed. The three new session tests cover
defaults, supported/unsupported values, connector snapshots, isolation,
and forwarding.
- `mvn -f fe/pom.xml -pl :fe-connector-mongodb -am test`: 35 connector
tests passed, including URI precedence, independent policy inheritance,
incompatible routing criteria, and stable transport configuration
identity.
- `build.sh --fe -j12`: passed. FE core and MongoDB connector
Checkstyle: zero violations.
- Generated `test_mongodb_read_policy.out` using the regression runner,
checked its results against the synthetic fixture, then independently
reran `test_mongodb_native` and `test_mongodb_read_policy`: 2 suites
passed, 0 failed/fatal/skipped. Covers SET/SET_VAR, all supported
modes/levels, invalid values, failed-hint restoration, URI policy
inheritance, and recovery.
- Additional replica-set integration: source profiler verified
primary/secondary selection and the actual read concern for 8 sequential
reads and 32 concurrent reads across 4 isolated sessions. Also verified
URI tags with a mode override and policy restoration after a failed
hint. This was an additional local acceptance check; the committed SQL
suites use the existing standalone fixture.
### Release note
Add `mongodb_read_preference` and `mongodb_read_concern` session
variables for native MongoDB queries, with statement-level SET_VAR
support. Both default to `inherit` to retain the catalog's existing read
policy.
### Check List (For Author)
- Behavior changed: Yes (see Details; explicit overrides now affect
native MongoDB data reads).
- Does this need documentation: Yes; the connector README is updated in
this PR.
- Pre-submission status: Ready. Local checks passed; CI is pending.
### AI Disclosure
- Tool: Codex (GPT-6).
- Generated: Implementation, tests, connector documentation, and this
description.
- Human review: Maintainer review pending; the agent completed a full
diff self-review and local validation.
---
fe/fe-connector/fe-connector-mongodb/README.md | 38 +++++++-
.../connector/mongodb/MongoCatalogProperties.java | 33 +++++++
.../connector/mongodb/MongoScanPlanProvider.java | 6 +-
.../doris/connector/mongodb/MongoScanPlanTest.java | 96 ++++++++++++++++++-
.../java/org/apache/doris/qe/SessionVariable.java | 31 +++++++
.../qe/SessionVariableMongoReadPolicyTest.java | 83 +++++++++++++++++
.../mongodb/test_mongodb_read_policy.out | 62 +++++++++++++
.../mongodb/test_mongodb_read_policy.groovy | 102 +++++++++++++++++++++
8 files changed, 443 insertions(+), 8 deletions(-)
diff --git a/fe/fe-connector/fe-connector-mongodb/README.md
b/fe/fe-connector/fe-connector-mongodb/README.md
index 5edf1b2ca49..f81d1e08392 100644
--- a/fe/fe-connector/fe-connector-mongodb/README.md
+++ b/fe/fe-connector/fe-connector-mongodb/README.md
@@ -98,9 +98,36 @@ find/getMore batches. Projection is materialized locally,
and every SQL filter,
LIKE/REGEXP, join, aggregation, sort and LIMIT remains a Doris operation. Range
splitting, predicate/LIMIT pushdown and shared snapshots are separate work.
-`batch_size` and `query_timeout` are session variables. URI read preference
settings
-take precedence over the catalog mode, retaining ordered tag sets and maximum
staleness
-in both drivers. Without URI routing settings, `mongodb.read_preference`
supplies the mode.
+`batch_size` and `query_timeout` are session variables. The following session
variables
+also support statement-level `SET_VAR` hints; values are case-insensitive:
+
+| Session variable | Default | Supported values |
+|---|---|---|
+| `mongodb_read_preference` | `inherit` | `inherit`, `primary`,
`primaryPreferred`, `secondary`, `secondaryPreferred`, `nearest` |
+| `mongodb_read_concern` | `inherit` | `inherit`, `local`, `majority`,
`available` |
+
+`inherit` uses the catalog's effective policy: URI `readPreference` /
`readConcernLevel`
+first, then `mongodb.read_preference` / `mongodb.read_concern`. An explicit
session or
+statement value overrides that policy for data reads. Each scan pins its
effective
+values without changing the catalog, metadata discovery, or other sessions.
+URI routing criteria (ordered tag sets, maximum staleness and hedge options)
remain
+in effect when overriding the mode. A `primary` override with incompatible URI
+criteria fails at planning instead of silently dropping those criteria.
+
+```sql
+SET mongodb_read_preference = 'secondaryPreferred';
+SET mongodb_read_concern = 'local';
+SELECT /*+ SET_VAR(mongodb_read_preference='primary',
mongodb_read_concern='majority') */
+ count(*) FROM mongo_native.sample.records;
+SET mongodb_read_preference = 'inherit';
+SET mongodb_read_concern = 'inherit';
+```
+
+Selecting a secondary can return stale data. `primaryPreferred` and
+`secondaryPreferred` permit fallback to the other role; `secondary` requires a
+secondary in a replica set. A read concern controls read visibility
independently
+of node selection and does not establish a snapshot across Doris scans.
+
Catalog connection defaults:
| Property | Default |
@@ -131,8 +158,9 @@ bash thirdparty/build-thirdparty.sh mongo_c_driver
FE unit tests reside in this module, BSON SerDe unit tests in
`be/test/core/data_type_serde/data_type_serde_bson_test.cpp`, and reader
lifecycle tests
-in `be/test/format_v2/table/mongodb_reader_test.cpp`. The regression suite
-`external_table_p0/mongodb/test_mongodb_native.groovy` uses the disposable
MongoDB
+in `be/test/format_v2/table/mongodb_reader_test.cpp`. The regression suites
+`external_table_p0/mongodb/test_mongodb_native.groovy` and
+`external_table_p0/mongodb/test_mongodb_read_policy.groovy` use the disposable
MongoDB
fixture under `docker/thirdparties/docker-compose/mongodb/`, enabled with
`enableMongoDBTest`, `externalEnvIp`, `mongodb_port`, `mongodb_user`, and
`mongodb_password` regression settings.
diff --git
a/fe/fe-connector/fe-connector-mongodb/src/main/java/org/apache/doris/connector/mongodb/MongoCatalogProperties.java
b/fe/fe-connector/fe-connector-mongodb/src/main/java/org/apache/doris/connector/mongodb/MongoCatalogProperties.java
index 64211845065..31478f3a8e1 100644
---
a/fe/fe-connector/fe-connector-mongodb/src/main/java/org/apache/doris/connector/mongodb/MongoCatalogProperties.java
+++
b/fe/fe-connector/fe-connector-mongodb/src/main/java/org/apache/doris/connector/mongodb/MongoCatalogProperties.java
@@ -26,6 +26,7 @@ import com.mongodb.MongoCredential;
import com.mongodb.ReadConcern;
import com.mongodb.ReadConcernLevel;
import com.mongodb.ReadPreference;
+import org.bson.BsonDocument;
import org.bson.Document;
import java.nio.charset.StandardCharsets;
@@ -144,6 +145,38 @@ public final class MongoCatalogProperties {
}
}
+ String resolveReadPreference(String policy) {
+ if ("inherit".equalsIgnoreCase(policy)) {
+ return readPreference;
+ }
+ String mode;
+ try {
+ mode = ReadPreference.valueOf(policy).getName();
+ } catch (IllegalArgumentException e) {
+ throw new IllegalArgumentException("Invalid
mongodb_read_preference");
+ }
+ if ("primary".equals(mode) && connectionString.getReadPreference() !=
null) {
+ BsonDocument inherited =
connectionString.getReadPreference().toDocument();
+ // BE preserves URI routing criteria when applying the statement's
mode. Reject
+ // incompatible overrides here rather than silently dropping the
catalog's routing.
+ if ((inherited.containsKey("tags") &&
!inherited.getArray("tags").isEmpty())
+ || inherited.containsKey("maxStalenessSeconds") ||
inherited.containsKey("hedge")) {
+ throw new
IllegalArgumentException("mongodb_read_preference=primary is incompatible with
URI "
+ + "read preference tags, max staleness or hedge
options");
+ }
+ }
+ return mode;
+ }
+
+ String resolveReadConcern(String policy) {
+ if ("inherit".equalsIgnoreCase(policy)) {
+ return readConcern;
+ }
+ String level = policy.toLowerCase(Locale.ROOT);
+ require(Set.of("local", "majority", "available").contains(level),
"mongodb_read_concern");
+ return level;
+ }
+
MongoClientSettings clientSettings() {
MongoClientSettings.Builder builder =
MongoClientSettings.builder().applyConnectionString(connectionString)
.applyToSocketSettings(s -> s.connectTimeout(connectTimeoutMs,
TimeUnit.MILLISECONDS)
diff --git
a/fe/fe-connector/fe-connector-mongodb/src/main/java/org/apache/doris/connector/mongodb/MongoScanPlanProvider.java
b/fe/fe-connector/fe-connector-mongodb/src/main/java/org/apache/doris/connector/mongodb/MongoScanPlanProvider.java
index 3aa9ecb2b0f..da9a76387fc 100644
---
a/fe/fe-connector/fe-connector-mongodb/src/main/java/org/apache/doris/connector/mongodb/MongoScanPlanProvider.java
+++
b/fe/fe-connector/fe-connector-mongodb/src/main/java/org/apache/doris/connector/mongodb/MongoScanPlanProvider.java
@@ -60,10 +60,12 @@ final class MongoScanPlanProvider implements
ConnectorScanPlanProvider {
d.setUsername(properties.getUsername());
d.setPassword(properties.getPassword());
d.setAuthDatabase(properties.getAuthDatabase());
- d.setReadPreference(properties.getReadPreference());
- d.setReadConcern(properties.getReadConcern());
d.setTypeMismatchAsNull("null".equals(properties.getTypeMismatch()));
Map<String, String> policies = session.getSessionProperties();
+ // Pin effective policies in the range; never mutate the catalog or
its shared metadata client.
+ d.setReadPreference(properties.resolveReadPreference(
+ policies.getOrDefault("mongodb_read_preference", "inherit")));
+
d.setReadConcern(properties.resolveReadConcern(policies.getOrDefault("mongodb_read_concern",
"inherit")));
d.setBatchSize(bounded(policies, "batch_size", 1024, 1, 65536));
d.setTimeoutMs(1000L * bounded(policies, "query_timeout", 300, 1,
Integer.MAX_VALUE / 1000));
d.setConnectTimeoutMs(properties.getConnectTimeoutMs());
diff --git
a/fe/fe-connector/fe-connector-mongodb/src/test/java/org/apache/doris/connector/mongodb/MongoScanPlanTest.java
b/fe/fe-connector/fe-connector-mongodb/src/test/java/org/apache/doris/connector/mongodb/MongoScanPlanTest.java
index 786544ade6a..1c07f6c78b4 100644
---
a/fe/fe-connector/fe-connector-mongodb/src/test/java/org/apache/doris/connector/mongodb/MongoScanPlanTest.java
+++
b/fe/fe-connector/fe-connector-mongodb/src/test/java/org/apache/doris/connector/mongodb/MongoScanPlanTest.java
@@ -22,6 +22,7 @@ import org.apache.doris.connector.spi.scan.ConnectorScanRange;
import org.apache.doris.connector.spi.scan.ConnectorScanRequest;
import org.apache.doris.connector.spi.scan.ScanNodePropertiesResult;
import org.apache.doris.thrift.TFileRangeDesc;
+import org.apache.doris.thrift.TMongoScanDesc;
import org.apache.doris.thrift.TTableFormatFileDesc;
import org.bson.Document;
@@ -34,6 +35,16 @@ import java.util.Optional;
class MongoScanPlanTest {
private static final class Session implements ConnectorSession {
+ private final Map<String, String> policies;
+
+ Session() {
+ this(Map.of("batch_size", "128", "query_timeout", "30"));
+ }
+
+ Session(Map<String, String> policies) {
+ this.policies = policies;
+ }
+
@Override
public String getQueryId() {
return "fixture-query";
@@ -76,7 +87,7 @@ class MongoScanPlanTest {
@Override
public Map<String, String> getSessionProperties() {
- return Map.of("batch_size", "128", "query_timeout", "30");
+ return policies;
}
}
@@ -85,6 +96,89 @@ class MongoScanPlanTest {
"mongodb.username", "fixture-user", "mongodb.password",
"fixture-secret"));
}
+ private static TMongoScanDesc plan(MongoCatalogProperties properties,
Map<String, String> policies) {
+ MongoTableHandle handle = new MongoTableHandle("sample",
+ new Document("name", "records").append("type", "collection"),
properties);
+ ConnectorScanRange range = new
MongoScanPlanProvider(properties).planScan(new Session(policies),
+ ConnectorScanRequest.builder(handle,
List.of()).build()).get(0);
+ TTableFormatFileDesc descriptor = new TTableFormatFileDesc();
+ range.populateRangeParams(descriptor, new TFileRangeDesc());
+ return descriptor.getMongodbParams();
+ }
+
+ @Test
+ void queryPoliciesOverrideCatalogAndRemainPinnedToTheirScan() {
+ MongoCatalogProperties properties = properties();
+ TMongoScanDesc overridden = plan(properties,
Map.of("mongodb_read_preference", "SECONDARYPREFERRED",
+ "mongodb_read_concern", "LOCAL"));
+ Assertions.assertEquals("secondaryPreferred",
overridden.getReadPreference());
+ Assertions.assertEquals("local", overridden.getReadConcern());
+ TMongoScanDesc inherited = plan(properties,
Map.of("mongodb_read_preference", "inherit",
+ "mongodb_read_concern", "inherit"));
+ Assertions.assertEquals("primary", inherited.getReadPreference());
+ Assertions.assertEquals("majority", inherited.getReadConcern());
+ Assertions.assertEquals("secondaryPreferred",
overridden.getReadPreference());
+ Assertions.assertEquals(inherited.getConfigurationVersion(),
overridden.getConfigurationVersion());
+ }
+
+ @Test
+ void inheritsUriPoliciesAndRejectsPrimaryWithSecondaryCriteria() {
+ MongoCatalogProperties properties =
MongoCatalogProperties.of(Map.of("mongodb.uri",
+
"mongodb://localhost:27017/?readPreference=secondary&readPreferenceTags=zone:east"
+ +
"&maxStalenessSeconds=120&readConcernLevel=available"));
+ TMongoScanDesc inherited = plan(properties, Map.of());
+ Assertions.assertEquals("secondary", inherited.getReadPreference());
+ Assertions.assertEquals("available", inherited.getReadConcern());
+ TMongoScanDesc overridden = plan(properties,
Map.of("mongodb_read_preference", "nearest",
+ "mongodb_read_concern", "majority"));
+ Assertions.assertEquals("nearest", overridden.getReadPreference());
+ Assertions.assertEquals(properties.getUri(), overridden.getUri());
+ Assertions.assertEquals("majority", overridden.getReadConcern());
+ Assertions.assertThrows(IllegalArgumentException.class,
+ () -> plan(properties, Map.of("mongodb_read_preference",
"primary")));
+ }
+
+ @Test
+ void
validatesEachInheritedRoutingConstraintWithoutChangingCatalogSettings() {
+ for (String suffix : List.of("&readPreferenceTags=zone:east",
"&maxStalenessSeconds=120")) {
+ MongoCatalogProperties properties =
MongoCatalogProperties.of(Map.of("mongodb.uri",
+ "mongodb://localhost:27017/?readPreference=secondary" +
suffix));
+ Assertions.assertThrows(IllegalArgumentException.class,
+ () -> plan(properties, Map.of("mongodb_read_preference",
"primary")));
+ Assertions.assertEquals("secondary", plan(properties,
Map.of()).getReadPreference());
+ }
+ MongoCatalogProperties modeOnly =
MongoCatalogProperties.of(Map.of("mongodb.uri",
+
"mongodb://localhost:27017/?readPreference=secondary&maxStalenessSeconds=-1"));
+ Assertions.assertEquals("primary",
+ plan(modeOnly, Map.of("mongodb_read_preference",
"primary")).getReadPreference());
+ MongoCatalogProperties catalog =
MongoCatalogProperties.of(Map.of("mongodb.uri", "mongodb://localhost:27017",
+ "mongodb.read_preference", "nearest", "mongodb.read_concern",
"local"));
+ TMongoScanDesc inherited = plan(catalog,
Map.of("mongodb_read_preference", "INHERIT",
+ "mongodb_read_concern", "INHERIT"));
+ Assertions.assertEquals("nearest", inherited.getReadPreference());
+ Assertions.assertEquals("local", inherited.getReadConcern());
+ }
+
+ @Test
+ void rejectsUnsupportedQueryPoliciesBeforeDispatch() {
+ for (String preference : List.of("", "invalid",
"secondary_preferred")) {
+ Assertions.assertThrows(IllegalArgumentException.class,
+ () -> plan(properties(), Map.of("mongodb_read_preference",
preference)));
+ }
+ for (String concern : List.of("", "invalid", "snapshot",
"linearizable")) {
+ Assertions.assertThrows(IllegalArgumentException.class,
+ () -> plan(properties(), Map.of("mongodb_read_concern",
concern)));
+ }
+ for (String preference : List.of("primary", "primaryPreferred",
"secondary", "secondaryPreferred", "nearest")) {
+ Assertions.assertEquals(preference,
+ plan(properties(), Map.of("mongodb_read_preference",
preference)).getReadPreference());
+ }
+ for (String concern : List.of("local", "majority", "available")) {
+ Assertions.assertEquals(concern,
+ plan(properties(), Map.of("mongodb_read_concern",
concern)).getReadConcern());
+ }
+ }
+
@Test
void emitsOneStableNativeRangeForEachLogicalObjectKind() {
MongoCatalogProperties properties = properties();
diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java
b/fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java
index cec3e1802f4..cbff9c43817 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java
@@ -3092,6 +3092,22 @@ public class SessionVariable implements Serializable,
Writable {
description = "enable count(*) pushdown optimization for external
table")
private boolean enableCountPushDownForExternalTable = true;
+ public static final String MONGODB_READ_PREFERENCE =
"mongodb_read_preference";
+ public static final String MONGODB_READ_CONCERN = "mongodb_read_concern";
+
+ @VarAttrDef.VarAttr(name = MONGODB_READ_PREFERENCE, needForward = true,
+ checker = "checkMongoDBReadPreference",
+ options = {"inherit", "primary", "primaryPreferred", "secondary",
"secondaryPreferred", "nearest"},
+ description = "MongoDB data-read routing for this session or
SET_VAR statement. "
+ + "inherit follows the catalog policy, including URI
routing criteria")
+ public String mongodbReadPreference = "inherit";
+
+ @VarAttrDef.VarAttr(name = MONGODB_READ_CONCERN, needForward = true,
+ checker = "checkMongoDBReadConcern", options = {"inherit",
"local", "majority", "available"},
+ description = "MongoDB data-read visibility for this session or
SET_VAR statement. "
+ + "inherit follows the catalog policy; shared snapshots
are not supported")
+ public String mongodbReadConcern = "inherit";
+
public static final String FLUSS_UNION_READ_MODE = "fluss_union_read_mode";
@VarAttrDef.VarAttr(name = FLUSS_UNION_READ_MODE,
@@ -4564,6 +4580,21 @@ public class SessionVariable implements Serializable,
Writable {
}
}
+ public void checkMongoDBReadPreference(String value) {
+ if (!Set.of("inherit", "primary", "primarypreferred", "secondary",
"secondarypreferred", "nearest")
+ .contains(value.toLowerCase(Locale.ROOT))) {
+ throw new UnsupportedOperationException("Invalid
mongodb_read_preference; expected inherit, primary, "
+ + "primaryPreferred, secondary, secondaryPreferred or
nearest");
+ }
+ }
+
+ public void checkMongoDBReadConcern(String value) {
+ if (!Set.of("inherit", "local", "majority",
"available").contains(value.toLowerCase(Locale.ROOT))) {
+ throw new UnsupportedOperationException("Invalid
mongodb_read_concern; expected inherit, local, "
+ + "majority or available");
+ }
+ }
+
public void checkFlussUnionReadMode(String mode) {
String normalized =
Strings.nullToEmpty(mode).trim().toLowerCase(Locale.ROOT);
if (!normalized.isEmpty() && !"auto".equals(normalized)
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/qe/SessionVariableMongoReadPolicyTest.java
b/fe/fe-core/src/test/java/org/apache/doris/qe/SessionVariableMongoReadPolicyTest.java
new file mode 100644
index 00000000000..927f844e5a3
--- /dev/null
+++
b/fe/fe-core/src/test/java/org/apache/doris/qe/SessionVariableMongoReadPolicyTest.java
@@ -0,0 +1,83 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.qe;
+
+import org.apache.doris.analysis.SetType;
+import org.apache.doris.analysis.SetVar;
+import org.apache.doris.analysis.StringLiteral;
+import org.apache.doris.common.DdlException;
+import org.apache.doris.connector.ConnectorSessionBuilder;
+import org.apache.doris.connector.spi.ConnectorSession;
+
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.util.List;
+import java.util.Map;
+
+class SessionVariableMongoReadPolicyTest {
+ private static void set(SessionVariable variables, String name, String
value) throws DdlException {
+ VariableMgr.setVar(variables, new SetVar(SetType.SESSION, name, new
StringLiteral(value)));
+ }
+
+ @Test
+ void defaultsInheritAndConnectorSnapshotsAreSessionIsolated() throws
Exception {
+ ConnectContext context = new ConnectContext();
+ SessionVariable variables = context.getSessionVariable();
+ ConnectorSession inherited =
ConnectorSessionBuilder.from(context).build();
+ Assertions.assertEquals("inherit",
inherited.getSessionProperties().get("mongodb_read_preference"));
+ Assertions.assertEquals("inherit",
inherited.getSessionProperties().get("mongodb_read_concern"));
+ set(variables, "mongodb_read_preference", "secondaryPreferred");
+ set(variables, "mongodb_read_concern", "local");
+ ConnectorSession overridden =
ConnectorSessionBuilder.from(context).build();
+ Assertions.assertEquals("secondaryPreferred",
overridden.getSessionProperties().get("mongodb_read_preference"));
+ Assertions.assertEquals("local",
overridden.getSessionProperties().get("mongodb_read_concern"));
+ Assertions.assertEquals("inherit",
inherited.getSessionProperties().get("mongodb_read_preference"));
+ Assertions.assertEquals("inherit", VariableMgr.toMap(new
SessionVariable()).get("mongodb_read_concern"));
+ }
+
+ @Test
+ void acceptsSupportedValuesAndForwardsPolicies() throws Exception {
+ SessionVariable variables = new SessionVariable();
+ for (String value : List.of("inherit", "primary", "primaryPreferred",
"secondary", "secondaryPreferred",
+ "nearest", "SECONDARY", "INHERIT")) {
+ set(variables, "mongodb_read_preference", value);
+ Assertions.assertEquals(value,
VariableMgr.toMap(variables).get("mongodb_read_preference"));
+ }
+ for (String value : List.of("inherit", "local", "majority",
"available", "LOCAL", "INHERIT")) {
+ set(variables, "mongodb_read_concern", value);
+ Assertions.assertEquals(value,
VariableMgr.toMap(variables).get("mongodb_read_concern"));
+ }
+ Map<String, String> forwarded = variables.getForwardVariables();
+ Assertions.assertEquals("INHERIT",
forwarded.get("mongodb_read_preference"));
+ Assertions.assertEquals("INHERIT",
forwarded.get("mongodb_read_concern"));
+ }
+
+ @Test
+ void rejectsUnsupportedValuesWithoutChangingSession() {
+ SessionVariable variables = new SessionVariable();
+ for (String value : List.of("", "invalid", "secondary_preferred", "
primary")) {
+ Assertions.assertThrows(DdlException.class, () -> set(variables,
"mongodb_read_preference", value));
+ }
+ for (String value : List.of("", "snapshot", "linearizable", "unknown",
" local")) {
+ Assertions.assertThrows(DdlException.class, () -> set(variables,
"mongodb_read_concern", value));
+ }
+ Assertions.assertEquals("inherit",
VariableMgr.toMap(variables).get("mongodb_read_preference"));
+ Assertions.assertEquals("inherit",
VariableMgr.toMap(variables).get("mongodb_read_concern"));
+ }
+}
diff --git
a/regression-test/data/external_table_p0/mongodb/test_mongodb_read_policy.out
b/regression-test/data/external_table_p0/mongodb/test_mongodb_read_policy.out
new file mode 100644
index 00000000000..a5f83be92dc
--- /dev/null
+++
b/regression-test/data/external_table_p0/mongodb/test_mongodb_read_policy.out
@@ -0,0 +1,62 @@
+-- This file is automatically generated. You should know what you did if you
want to edit this
+-- !defaults --
+inherit inherit
+
+-- !show_variables --
+mongodb_read_concern inherit inherit 0
+mongodb_read_preference inherit inherit 0
+
+-- !catalog_defaults --
+4 10
+
+-- !mode_primary --
+4 10
+
+-- !mode_primaryPreferred --
+4 10
+
+-- !mode_secondary --
+4 10
+
+-- !mode_secondaryPreferred --
+4 10
+
+-- !mode_nearest --
+4 10
+
+-- !concern_local --
+4 10
+
+-- !concern_majority --
+4 10
+
+-- !concern_available --
+4 10
+
+-- !hint --
+secondaryPreferred local 4 10
+
+-- !after_hint --
+primary majority
+
+-- !hint_inherit --
+4 10
+
+-- !after_inherit_hint --
+primary majority
+
+-- !after_invalid --
+primary majority
+
+-- !uri_inherit --
+4
+
+-- !after_failed_hint --
+inherit inherit
+
+-- !hint_recover --
+4
+
+-- !after_recover --
+primary inherit
+
diff --git
a/regression-test/suites/external_table_p0/mongodb/test_mongodb_read_policy.groovy
b/regression-test/suites/external_table_p0/mongodb/test_mongodb_read_policy.groovy
new file mode 100644
index 00000000000..11af0133457
--- /dev/null
+++
b/regression-test/suites/external_table_p0/mongodb/test_mongodb_read_policy.groovy
@@ -0,0 +1,102 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_mongodb_read_policy", "p0,external") {
+ if
(!"true".equalsIgnoreCase(context.config.otherConfigs.get("enableMongoDBTest")))
{
+ return
+ }
+ String host = context.config.otherConfigs.get("externalEnvIp")
+ String port = context.config.otherConfigs.get("mongodb_port")
+ String user = context.config.otherConfigs.get("mongodb_user")
+ String password = context.config.otherConfigs.get("mongodb_password")
+ def original = sql("SELECT @@mongodb_read_preference,
@@mongodb_read_concern")[0]
+ def create = { String name, String uri ->
+ sql "DROP CATALOG IF EXISTS ${name}"
+ def properties = ["type": "mongodb", "mongodb.uri": uri,
"mongodb.username": user,
+ "mongodb.password": password, "mongodb.databases":
"doris_mongodb_native",
+ "mongodb.read_preference": "secondaryPreferred",
"mongodb.read_concern": "local"]
+ String values = properties.collect { key, value ->
+ "\"${key}\" = '${value.toString().replace("\\",
"\\\\").replace("'", "\\'")}'"
+ }.join(",\n")
+ sql "CREATE CATALOG ${name} PROPERTIES (${values})"
+ }
+ try {
+ sql "SET mongodb_read_preference = 'inherit'"
+ sql "SET mongodb_read_concern = 'inherit'"
+ qt_defaults "SELECT @@mongodb_read_preference, @@mongodb_read_concern"
+ order_qt_show_variables "SHOW VARIABLES LIKE 'mongodb_read_%'"
+ create("mongodb_read_policy", "mongodb://${host}:${port}")
+ qt_catalog_defaults "SELECT count(*), sum(seq) FROM
mongodb_read_policy.doris_mongodb_native.typed_records"
+ for (String mode : ["primary", "primaryPreferred", "secondary",
"secondaryPreferred", "nearest"]) {
+ sql "SET mongodb_read_preference = '${mode}'"
+ "qt_mode_${mode}" "SELECT count(*), sum(seq) FROM
mongodb_read_policy.doris_mongodb_native.typed_records"
+ }
+ for (String level : ["local", "majority", "available"]) {
+ sql "SET mongodb_read_concern = '${level}'"
+ "qt_concern_${level}" "SELECT count(*), sum(seq) FROM
mongodb_read_policy.doris_mongodb_native.typed_records"
+ }
+ sql "SET mongodb_read_preference = 'primary'"
+ sql "SET mongodb_read_concern = 'majority'"
+ qt_hint """SELECT /*+
SET_VAR(mongodb_read_preference='secondaryPreferred',
mongodb_read_concern='local') */
+ @@mongodb_read_preference, @@mongodb_read_concern, count(*),
sum(seq)
+ FROM mongodb_read_policy.doris_mongodb_native.typed_records"""
+ qt_after_hint "SELECT @@mongodb_read_preference,
@@mongodb_read_concern"
+ qt_hint_inherit """SELECT /*+
SET_VAR(mongodb_read_preference='inherit', mongodb_read_concern='inherit') */
+ count(*), sum(seq) FROM
mongodb_read_policy.doris_mongodb_native.typed_records"""
+ qt_after_inherit_hint "SELECT @@mongodb_read_preference,
@@mongodb_read_concern"
+ for (String mode : ["invalid", "secondary_preferred", ""]) {
+ test {
+ sql "SET mongodb_read_preference = '${mode}'"
+ exception "mongodb_read_preference"
+ }
+ }
+ for (String level : ["snapshot", "linearizable", "invalid", ""]) {
+ test {
+ sql "SET mongodb_read_concern = '${level}'"
+ exception "mongodb_read_concern"
+ }
+ }
+ test {
+ sql """SELECT /*+ SET_VAR(mongodb_read_concern='snapshot') */
+ count(*) FROM
mongodb_read_policy.doris_mongodb_native.typed_records"""
+ exception "mongodb_read_concern"
+ }
+ qt_after_invalid "SELECT @@mongodb_read_preference,
@@mongodb_read_concern"
+ create("mongodb_read_policy_uri",
"mongodb://${host}:${port}/?readPreference=secondaryPreferred"
+ +
"&readPreferenceTags=zone:east&maxStalenessSeconds=120&readConcernLevel=available")
+ sql "SET mongodb_read_preference = 'inherit'"
+ sql "SET mongodb_read_concern = 'inherit'"
+ qt_uri_inherit "SELECT count(*) FROM
mongodb_read_policy_uri.doris_mongodb_native.typed_records"
+ test {
+ sql """SELECT /*+ SET_VAR(mongodb_read_preference='primary') */
+ count(*) FROM
mongodb_read_policy_uri.doris_mongodb_native.typed_records"""
+ exception "incompatible with URI"
+ }
+ qt_after_failed_hint "SELECT @@mongodb_read_preference,
@@mongodb_read_concern"
+ sql "SET mongodb_read_preference = 'primary'"
+ test {
+ sql "SELECT count(*) FROM
mongodb_read_policy_uri.doris_mongodb_native.typed_records"
+ exception "incompatible with URI"
+ }
+ qt_hint_recover """SELECT /*+
SET_VAR(mongodb_read_preference='secondaryPreferred',
mongodb_read_concern='local') */
+ count(*) FROM
mongodb_read_policy_uri.doris_mongodb_native.typed_records"""
+ qt_after_recover "SELECT @@mongodb_read_preference,
@@mongodb_read_concern"
+ } finally {
+ sql "SET mongodb_read_preference = '${original[0]}'"
+ sql "SET mongodb_read_concern = '${original[1]}'"
+ }
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]