This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new d420f9c41bb [fix](fe) Load FE images that carry Lance catalogs written
by 4.1.4+ (#69575)
d420f9c41bb is described below
commit d420f9c41bb657c661231f1f4274ac7aae19f0a7
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Thu Oct 29 05:04:08 2026 +0000
[fix](fe) Load FE images that carry Lance catalogs written by 4.1.4+
(#69575)
### What problem does this PR solve?
Issue Number: None
Related PR: #65730, #67235, #68781
Problem Summary:
**In short.** This PR makes master read the Lance catalogs that
branch-4.1 persists (#65730, released in 4.1.4): an FE image written by
4.1.4 or later with a Lance catalog in it now loads. Before this PR the
image failed to parse, so a cluster with a Lance catalog could not be
upgraded from any released 4.1.4 / 4.1.4.1 / 4.1.5 build (or from 4.2)
to a master-based build: the FE did not start. The Lance catalog's GSON
labels are mapped to the plugin-driven classes, as the Iceberg, Paimon
and LakeSoul labels already are, and the catalog stays registered,
unusable until a Lance connector plugin is installed.
**Background.** branch-4.1 added a Lance catalog as fe-core classes in
#65730 (released in 4.1.4). Like every external catalog, it is persisted
in the image's `datasource` module as GSON JSON whose `clazz` field
names the class, `LanceExternalCatalog`, with `logType` `LANCE`. master
has no Lance classes: external catalogs are served by connector plugins,
and the labels of the catalogs that used to be fe-core classes are
redirected to `PluginDrivenExternalCatalog` /
`PluginDrivenExternalDatabase` / `PluginDrivenMvccExternalTable` by
`registerCompatibleSubtype` in `GsonUtils`. A Lance connector plugin
does not exist on master yet; it will be ported later.
branch-4.1 also had, for a while, a durable Lance index job framework
(#67235) that wrote edit-log op 500 (`OP_LANCE_INDEX_JOB_UPSERT`) and an
image module named `lanceIndexJobManager`. #68781 reverted it on
branch-4.1 and branch-4.2 before any release: among the release tags
only 4.1.5-rc01 has the writer, while 4.1.5 and 4.2.0-rc01 keep just a
tombstone that reads and discards both. master never had the framework,
and this PR does not make it read that metadata either.
The upgrade fixtures on master (`upgrade/413`) were generated from
4.1.3, before Lance existed, so no test caught this.
**The problem, and what it cost.** Compared on master against fixtures
emitted by real 4.1.5 bytecode:
| Metadata from a 4.1.4+ FE | master before this PR | after this PR |
|---|---|---|
| image whose `datasource` module has a Lance catalog (every 4.1.4+
cluster with one) | image fails to parse: `cannot deserialize interface
org.apache.doris.datasource.CatalogIf subtype named
LanceExternalCatalog; did you forget to register a subtype?`; the FE
cannot start | loads; the catalog is a `PluginDrivenExternalCatalog` of
type `lance` with all its properties, comment and auto-analyze policies;
using it fails with `No ConnectorProvider found for plugin-driven
catalog: <name>, type: lance. Ensure the connector plugin is installed.`
|
| `OP_CREATE_CATALOG` of a Lance catalog | replays as a degraded catalog
(the entry carries only properties) | unchanged |
The only labels in the GSON registries of 4.1.4.1, 4.1.5 and 4.2.0-rc01
that master does not know are `LanceExternalCatalog`,
`LanceExternalDatabase` and `LanceExternalTable`.
**How this PR fixes it.**
- `GsonUtils`: `LanceExternalCatalog` → `PluginDrivenExternalCatalog`,
`LanceExternalDatabase` → `PluginDrivenExternalDatabase`,
`LanceExternalTable` → `PluginDrivenMvccExternalTable`
(`LanceExternalTable` implemented `MvccTable`). 4.1.x never persists an
external database or table, so only the catalog label can actually
appear; the other two complete the registry, as for the other migrated
connectors.
- `InitCatalogLog.Type` keeps `LANCE`, so the `logType` 4.1.4+ wrote
still deserializes to itself. Nothing depends on it: a Lance catalog
always persists its `type` property (no resource type creates one), and
`PluginDrivenExternalCatalog#gsonPostProcess` rewrites the logType to
`PLUGIN`.
- `BdbjeJournalDataProcNode`, which renders `SHOW PROC
'/bdbje/<db>/<journalId>'` in BDBJE debug mode, prints a record read
without a payload as `null`, as `BDBTool` already does, instead of
failing with a `NullPointerException`. This is an existing master issue
that came up in review, not a Lance one: the readers of the retired
operation types already on master (the PL/SQL ones of #58700 and the
external-catalog ones) keep no payload.
Until a Lance connector plugin is installed, an upgraded cluster lists
its Lance catalogs and fails only when one is used; installing the
plugin makes them work without any metadata change, since the catalog
properties are handed to the connector unchanged.
**Results.**
- New golden fixtures `fe/fe-core/src/test/resources/upgrade/415/`,
emitted by 4.1.5-release bytecode like `upgrade/413` (generator and
`PROVENANCE.txt` included, byte-deterministic): the `datasource` module
with three Lance catalogs (filesystem on S3, filesystem on `file://`,
REST with bearer auth), their `OP_CREATE_CATALOG` entries, and 4.1.5's
label registries.
- `Legacy415LanceUpgradeTest` (7 tests): the image loads and every Lance
catalog becomes a `PluginDrivenExternalCatalog` of type `lance` with the
`PLUGIN` logType; every persisted property, the comment,
`lastUpdateTime` and the auto-analyze policy survive; the persisted
logType names still deserialize; without a plugin the catalog fails only
when used, with the message above; a provider claiming `lance` receives
exactly the persisted properties, from the image and from the journal;
the journal entries replay with and without a plugin.
- `LanceGsonCompatReplayTest` (4 tests): the three labels replay as the
expected classes (the table as the MVCC variant), and they are the only
labels 4.1.5 registered beyond 4.1.3.
- Each of these changes, undone on its own, turns tests red: dropping
the catalog label (5 tests, with the parse error above), the database
label, mapping the table label to the non-MVCC class, or removing
`LANCE`.
- `BdbjeJournalDataProcNodeTest` (2 tests): a retired PL/SQL record, in
its written form, shows its journal ID, operation name, size and `null`
data in the debug view; a live `OP_SAVE_NEXTID` record still shows its
payload. With `getData().toString()` back, the first test fails with a
`NullPointerException`.
- The three classes above with 13 related upgrade, GSON and journal
suites (`Legacy413ImageMigrationTest`, `*GsonCompatReplayTest`,
`LegacyExternalTableTypeReplayTest`, `CatalogMgrTest`,
`ExternalCatalogTest`, `ConstraintPersistTest`, `BDBJEJournalTest`,
`BDBToolTest`, `BDBDebuggerTest`, ...): 103 tests pass; fe-core
checkstyle passes.
Co-authored-by: kid <[email protected]>
---
.licenserc.yaml | 7 +
.../common/proc/BdbjeJournalDataProcNode.java | 4 +-
.../doris/datasource/log/InitCatalogLog.java | 5 +
.../org/apache/doris/persist/gson/GsonUtils.java | 16 +-
.../common/proc/BdbjeJournalDataProcNodeTest.java | 121 ++++++++++
.../datasource/LanceGsonCompatReplayTest.java | 121 ++++++++++
.../apache/doris/datasource/Legacy413Fixtures.java | 46 ++--
.../apache/doris/datasource/Legacy415Fixtures.java | 74 ++++++
.../datasource/Legacy415LanceUpgradeTest.java | 227 +++++++++++++++++++
.../src/test/resources/upgrade/415/PROVENANCE.txt | 56 +++++
.../src/test/resources/upgrade/415/catalogMgr.json | 1 +
.../resources/upgrade/415/datasource.module.bin | Bin 0 -> 1302 bytes
.../src/test/resources/upgrade/415/editlog.index | 4 +
.../415/editlog/op320-create-g1_lance_local.bin | Bin 0 -> 222 bytes
.../415/editlog/op320-create-g1_lance_rest.bin | Bin 0 -> 364 bytes
.../415/editlog/op320-create-g1_lance_s3.bin | Bin 0 -> 354 bytes
.../upgrade/415/generator/Gen415Fixtures.java | 248 +++++++++++++++++++++
.../src/test/resources/upgrade/415/labels.db.txt | 15 ++
.../src/test/resources/upgrade/415/labels.ds.txt | 26 +++
.../src/test/resources/upgrade/415/labels.tbl.txt | 29 +++
20 files changed, 984 insertions(+), 16 deletions(-)
diff --git a/.licenserc.yaml b/.licenserc.yaml
index 4d2bf21c833..e28192e8df0 100644
--- a/.licenserc.yaml
+++ b/.licenserc.yaml
@@ -96,6 +96,13 @@ header:
- "fe/fe-core/src/test/resources/upgrade/413/labels.db.txt"
- "fe/fe-core/src/test/resources/upgrade/413/labels.ds.txt"
- "fe/fe-core/src/test/resources/upgrade/413/labels.tbl.txt"
+ # Same for the golden 4.1.5 (Lance catalog) fixtures emitted by
Gen415Fixtures;
+ # upgrade/415/generator/Gen415Fixtures.java stays checked.
+ - "fe/fe-core/src/test/resources/upgrade/415/PROVENANCE.txt"
+ - "fe/fe-core/src/test/resources/upgrade/415/editlog.index"
+ - "fe/fe-core/src/test/resources/upgrade/415/labels.db.txt"
+ - "fe/fe-core/src/test/resources/upgrade/415/labels.ds.txt"
+ - "fe/fe-core/src/test/resources/upgrade/415/labels.tbl.txt"
-
"fe/fe-core/src/main/java/software/amazon/awssdk/core/client/builder/SdkDefaultClientBuilder.java"
- "fe/fe-core/src/main/antlr4/org/apache/doris/nereids/JavaLexer.g4"
- "fe/fe-core/src/main/antlr4/org/apache/doris/nereids/JavaParser.g4"
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/common/proc/BdbjeJournalDataProcNode.java
b/fe/fe-core/src/main/java/org/apache/doris/common/proc/BdbjeJournalDataProcNode.java
index 6b18bd05a46..6f29d4a37d1 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/common/proc/BdbjeJournalDataProcNode.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/common/proc/BdbjeJournalDataProcNode.java
@@ -52,7 +52,9 @@ public class BdbjeJournalDataProcNode implements
ProcNodeInterface {
BDBDebugger.JournalEntityWrapper entity = env.getJournalEntity(dbName,
journalId);
short opCode = entity.entity == null ? -1 : entity.entity.getOpCode();
- String data = entity.entity == null ? entity.errMsg :
entity.entity.getData().toString();
+ // The readers of retired operation types (e.g. the PL/SQL ones)
consume the payload without keeping it;
+ // print such a record the way BDBTool does.
+ String data = entity.entity == null ? entity.errMsg :
String.valueOf(entity.entity.getData());
result.addRow(Lists.newArrayList(entity.journalId.toString(),
OperationType.getOpName(opCode), entity.size.toString(),
data));
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/datasource/log/InitCatalogLog.java
b/fe/fe-core/src/main/java/org/apache/doris/datasource/log/InitCatalogLog.java
index f97d5cd04c4..6ae6ab508f3 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/datasource/log/InitCatalogLog.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/datasource/log/InitCatalogLog.java
@@ -42,6 +42,10 @@ public class InitCatalogLog implements Writable {
*
* <p>HUDI is absent because it was never produced: hudi tables have
always lived in an hms catalog and
* no {@code HudiExternalCatalog} ever existed, so no image can carry
it.</p>
+ *
+ * <p>LANCE is written by the 4.1.4+ FEs that shipped a Lance catalog. A
Lance catalog always persists its
+ * {@code type} property (no resource type creates one), so nothing is
backfilled from it; it stays so
+ * that every name a released image carries still deserializes to
itself.</p>
*/
public enum Type {
HMS,
@@ -54,6 +58,7 @@ public class InitCatalogLog implements Writable {
TEST,
TRINO_CONNECTOR,
REMOTE_DORIS,
+ LANCE,
PLUGIN,
UNKNOWN;
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/persist/gson/GsonUtils.java
b/fe/fe-core/src/main/java/org/apache/doris/persist/gson/GsonUtils.java
index 264f98cbdeb..ad05a6e0f32 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/persist/gson/GsonUtils.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/persist/gson/GsonUtils.java
@@ -413,7 +413,12 @@ public class GsonUtils {
// was never migrated to a connector, so the remapped catalog
has no backing connector and
// errors on access (it must be dropped); this only keeps old
images/edit-logs loadable.
.registerCompatibleSubtype(
- PluginDrivenExternalCatalog.class,
"LakeSoulExternalCatalog");
+ PluginDrivenExternalCatalog.class,
"LakeSoulExternalCatalog")
+ // Migrate Lance catalogs written by 4.1.4+ (4.1.x / 4.2) to
PluginDriven on deserialization.
+ // Until a lance connector plugin is installed the catalog has
no backing connector and errors
+ // on access; without this label the whole image fails to
parse and the FE cannot start.
+ .registerCompatibleSubtype(
+ PluginDrivenExternalCatalog.class,
"LanceExternalCatalog");
if (Config.isNotCloudMode()) {
dsTypeAdapterFactory
.registerSubtype(InternalCatalog.class,
InternalCatalog.class.getSimpleName());
@@ -472,7 +477,10 @@ public class GsonUtils {
PluginDrivenExternalDatabase.class, "HMSExternalDatabase")
// Migrate old LakeSoul databases to PluginDriven on
deserialization (deprecated, no connector)
.registerCompatibleSubtype(
- PluginDrivenExternalDatabase.class,
"LakeSoulExternalDatabase");
+ PluginDrivenExternalDatabase.class,
"LakeSoulExternalDatabase")
+ // Migrate Lance databases (4.1.4+) to PluginDriven on
deserialization
+ .registerCompatibleSubtype(
+ PluginDrivenExternalDatabase.class,
"LanceExternalDatabase");
private static RuntimeTypeAdapterFactory<TableIf> tblTypeAdapterFactory =
RuntimeTypeAdapterFactory.of(
TableIf.class,
"clazz").registerSubtype(ExternalTable.class,
ExternalTable.class.getSimpleName())
@@ -508,6 +516,10 @@ public class GsonUtils {
// PluginDrivenMvccExternalTable — matching what
buildTableInternal rebuilds it as on replay.
.registerCompatibleSubtype(
PluginDrivenMvccExternalTable.class, "HMSExternalTable")
+ // Lance tables (4.1.4+) migrate to the MVCC variant:
LanceExternalTable implemented MvccTable
+ // (dataset versions back snapshots/time-travel)
+ .registerCompatibleSubtype(
+ PluginDrivenMvccExternalTable.class, "LanceExternalTable")
.registerSubtype(BrokerTable.class,
BrokerTable.class.getSimpleName())
.registerSubtype(EsTable.class, EsTable.class.getSimpleName())
.registerSubtype(FunctionGenTable.class,
FunctionGenTable.class.getSimpleName())
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/common/proc/BdbjeJournalDataProcNodeTest.java
b/fe/fe-core/src/test/java/org/apache/doris/common/proc/BdbjeJournalDataProcNodeTest.java
new file mode 100644
index 00000000000..95c7ac0d0bc
--- /dev/null
+++
b/fe/fe-core/src/test/java/org/apache/doris/common/proc/BdbjeJournalDataProcNodeTest.java
@@ -0,0 +1,121 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.common.proc;
+
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.common.Config;
+import org.apache.doris.common.io.Text;
+import org.apache.doris.common.jmockit.Deencapsulation;
+import org.apache.doris.journal.JournalEntity;
+import org.apache.doris.journal.bdbje.BDBDebugger;
+import org.apache.doris.persist.OperationType;
+
+import com.google.common.collect.Lists;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.mockito.Mockito;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.DataInputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.util.List;
+
+/**
+ * SHOW PROC '/bdbje/dbName/journalId', the BDBJE debug-mode view of a single
journal record.
+ */
+public class BdbjeJournalDataProcNodeTest {
+ private static final String DB_NAME = "1";
+
+ private boolean debugMode;
+ private BDBDebugger.BDBDebugEnv debugEnv;
+
+ @BeforeEach
+ public void setUp() {
+ debugMode = Config.enable_bdbje_debug_mode;
+ debugEnv = BDBDebugger.get().getEnv();
+ }
+
+ @AfterEach
+ public void tearDown() {
+ Config.enable_bdbje_debug_mode = debugMode;
+ Deencapsulation.setField(BDBDebugger.get(), "debugEnv", debugEnv);
+ }
+
+ @Test
+ public void recordOfARetiredOperationTypeIsPrintable() throws Exception {
+ // The readers of retired operation types consume the payload without
keeping it, e.g. the PL/SQL ones
+ // that #58700 retired, whose records are Text-framed JSON.
+ BDBDebugger.JournalEntityWrapper plsqlPackage = record(8L,
OperationType.OP_DROP_PLSQL_PACKAGE,
+ "{\"name\":\"pkg1\",\"catalogId\":0,\"dbName\":\"db1\"}");
+ Assertions.assertNull(plsqlPackage.entity.getData());
+ installDebugEnv(plsqlPackage);
+
+ // MUTATION: with getData().toString() the row fails with a
NullPointerException.
+ Assertions.assertEquals(Lists.newArrayList("8",
"OP_DROP_PLSQL_PACKAGE",
+ plsqlPackage.size.toString(), "null"), fetchRow(8L));
+ }
+
+ @Test
+ public void liveRecordStillShowsItsPayload() throws Exception {
+ BDBDebugger.JournalEntityWrapper nextId = record(9L,
OperationType.OP_SAVE_NEXTID, "12345");
+ installDebugEnv(nextId);
+
+ Assertions.assertEquals(Lists.newArrayList("9", "OP_SAVE_NEXTID",
nextId.size.toString(), "12345"),
+ fetchRow(9L));
+ }
+
+ /** A journal record as BDBDebugEnv#getJournalEntity reads it back from
BDBJE. */
+ private static BDBDebugger.JournalEntityWrapper record(long journalId,
short opCode, String payload)
+ throws IOException {
+ ByteArrayOutputStream bytes = new ByteArrayOutputStream();
+ try (DataOutputStream out = new DataOutputStream(bytes)) {
+ out.writeShort(opCode);
+ // The int-length framing that both Text#write and
Text.writeString produce.
+ Text.writeString(out, payload);
+ }
+ BDBDebugger.JournalEntityWrapper wrapper = new
BDBDebugger.JournalEntityWrapper(journalId);
+ wrapper.size = bytes.size();
+ JournalEntity entity = new JournalEntity();
+ try (DataInputStream in = new DataInputStream(new
ByteArrayInputStream(bytes.toByteArray()))) {
+ entity.readFields(in);
+ Assertions.assertEquals(-1, in.read(), "the record must be
consumed exactly");
+ }
+ wrapper.entity = entity;
+ return wrapper;
+ }
+
+ private static void installDebugEnv(BDBDebugger.JournalEntityWrapper...
records) {
+ BDBDebugger.BDBDebugEnv env =
Mockito.mock(BDBDebugger.BDBDebugEnv.class);
+ for (BDBDebugger.JournalEntityWrapper record : records) {
+ Mockito.when(env.getJournalEntity(DB_NAME,
record.journalId)).thenReturn(record);
+ }
+ Deencapsulation.setField(BDBDebugger.get(), "debugEnv", env);
+ Config.enable_bdbje_debug_mode = true;
+ }
+
+ private static List<String> fetchRow(long journalId) throws
AnalysisException {
+ ProcResult result = new BdbjeJournalDataProcNode(DB_NAME,
journalId).fetchResult();
+ Assertions.assertEquals(BdbjeJournalDataProcNode.TITLE_NAMES,
result.getColumnNames());
+ Assertions.assertEquals(1, result.getRows().size());
+ return result.getRows().get(0);
+ }
+}
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/datasource/LanceGsonCompatReplayTest.java
b/fe/fe-core/src/test/java/org/apache/doris/datasource/LanceGsonCompatReplayTest.java
new file mode 100644
index 00000000000..e54b03a759b
--- /dev/null
+++
b/fe/fe-core/src/test/java/org/apache/doris/datasource/LanceGsonCompatReplayTest.java
@@ -0,0 +1,121 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.datasource;
+
+import org.apache.doris.catalog.DatabaseIf;
+import org.apache.doris.catalog.TableIf;
+import org.apache.doris.datasource.mvcc.PluginDrivenMvccExternalTable;
+import org.apache.doris.datasource.plugin.PluginDrivenExternalCatalog;
+import org.apache.doris.datasource.plugin.PluginDrivenExternalDatabase;
+import org.apache.doris.persist.gson.GsonUtils;
+
+import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.Maps;
+import com.google.common.collect.Sets;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Guards loading the Lance catalogs/databases/tables that 4.1.4+ FEs
persisted (4.1.x and 4.2 shipped Lance as
+ * fe-core classes; this branch has no such classes). Their GSON "clazz"
labels must be redirected by the
+ * {@code registerCompatibleSubtype} mappings in {@link GsonUtils} to the
generic PluginDriven classes —
+ * otherwise an image from a cluster with a Lance catalog fails with a {@code
JsonParseException} (label not
+ * registered) and the FE cannot start.
+ *
+ * <p>The table label lands on the MVCC variant: {@code LanceExternalTable}
implemented {@code MvccTable}
+ * (dataset versions back snapshots and time travel), so replaying it as the
base
+ * {@code PluginDrivenExternalTable} would silently downgrade it.</p>
+ *
+ * <p>Each case round-trips a valid PluginDriven object through GSON and
rewrites only the "clazz" discriminator
+ * to the legacy label, like {@link PaimonGsonCompatReplayTest}. The real
4.1.5 bytes of a Lance catalog are
+ * covered by {@link Legacy415LanceUpgradeTest}; a database or table is never
persisted by 4.1.5.</p>
+ */
+public class LanceGsonCompatReplayTest {
+
+ private static String swapClazz(String json, String currentTag, String
legacyTag) {
+ String needle = "\"clazz\":\"" + currentTag + "\"";
+ // Sanity: the polymorphic serialization must emit the discriminator
we are about to rewrite.
+ Assertions.assertTrue(json.contains(needle),
+ "expected discriminator " + needle + " in serialized json: " +
json);
+ return json.replace(needle, "\"clazz\":\"" + legacyTag + "\"");
+ }
+
+ @Test
+ public void testLegacyLanceCatalogTagReplaysAsPluginDriven() {
+ Map<String, String> props = Maps.newHashMap();
+ props.put("type", "lance");
+ PluginDrivenExternalCatalog catalog =
+ new PluginDrivenExternalCatalog(1L, "lance_ctl", "", props,
"c", null);
+ String baseJson = GsonUtils.GSON.toJson(catalog, CatalogIf.class);
+
+ String json = swapClazz(baseJson, "PluginDrivenExternalCatalog",
"LanceExternalCatalog");
+ // MUTATION: removing the registerCompatibleSubtype throws "cannot
deserialize ... subtype named
+ // LanceExternalCatalog" here; a wrong target class fails the
assertSame.
+ CatalogIf<?> restored = GsonUtils.GSON.fromJson(json, CatalogIf.class);
+ Assertions.assertSame(PluginDrivenExternalCatalog.class,
restored.getClass(),
+ "legacy 'LanceExternalCatalog' tag must replay as
PluginDrivenExternalCatalog");
+ }
+
+ @Test
+ public void testLegacyLanceDatabaseTagReplaysAsPluginDriven() {
+ PluginDrivenExternalDatabase db = new PluginDrivenExternalDatabase();
+ db.id = 2L;
+ db.name = "lance_db";
+ String baseJson = GsonUtils.GSON.toJson(db, DatabaseIf.class);
+
+ String json = swapClazz(baseJson, "PluginDrivenExternalDatabase",
"LanceExternalDatabase");
+ // MUTATION: dropping the db registerCompatibleSubtype makes this
throw on deserialize.
+ DatabaseIf<?> restored = GsonUtils.GSON.fromJson(json,
DatabaseIf.class);
+ Assertions.assertSame(PluginDrivenExternalDatabase.class,
restored.getClass(),
+ "legacy 'LanceExternalDatabase' tag must replay as
PluginDrivenExternalDatabase");
+ }
+
+ @Test
+ public void testLegacyLanceTableTagReplaysAsMvccPluginDriven() {
+ PluginDrivenMvccExternalTable table = new
PluginDrivenMvccExternalTable();
+ table.id = 3L;
+ table.name = "lance_tbl";
+ table.dbName = "lance_db";
+ String baseJson = GsonUtils.GSON.toJson(table, TableIf.class);
+
+ String json = swapClazz(baseJson, "PluginDrivenMvccExternalTable",
"LanceExternalTable");
+ TableIf restored = GsonUtils.GSON.fromJson(json, TableIf.class);
+ // assertSame, not instanceof: a mapping to the base
PluginDrivenExternalTable would also pass instanceof.
+ Assertions.assertSame(PluginDrivenMvccExternalTable.class,
restored.getClass(),
+ "legacy 'LanceExternalTable' tag must replay as
PluginDrivenMvccExternalTable,"
+ + " not the base PluginDrivenExternalTable");
+ }
+
+ @Test
+ public void lanceLabelsAreTheOnlyOnesAReleaseAddedAfter413() throws
Exception {
+ // The three cases above are complete only if Lance is all that
4.1.5's registries added on top of
+ // 4.1.3's (which Legacy413LabelRegistryTest covers label by label).
Both inventories are dumped
+ // reflectively by the fixture generators, so a label added in a later
release cannot hide here.
+ Assertions.assertEquals(ImmutableSet.of("LanceExternalCatalog"),
addedSince413("ds"));
+ Assertions.assertEquals(ImmutableSet.of("LanceExternalDatabase"),
addedSince413("db"));
+ Assertions.assertEquals(ImmutableSet.of("LanceExternalTable"),
addedSince413("tbl"));
+ }
+
+ private static Set<String> addedSince413(String which) throws Exception {
+ return Sets.difference(Legacy415Fixtures.legacyLabels(which).keySet(),
+ Legacy413Fixtures.legacyLabels(which).keySet());
+ }
+}
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/datasource/Legacy413Fixtures.java
b/fe/fe-core/src/test/java/org/apache/doris/datasource/Legacy413Fixtures.java
index 5f8f2c33d3b..a9482611c95 100644
---
a/fe/fe-core/src/test/java/org/apache/doris/datasource/Legacy413Fixtures.java
+++
b/fe/fe-core/src/test/java/org/apache/doris/datasource/Legacy413Fixtures.java
@@ -34,6 +34,9 @@ import java.util.Map;
* <p>Those files were produced by real 4.1.3 bytecode, not written by hand;
see the PROVENANCE.txt
* next to them for how to regenerate and for the five wire-format details
that make hand-authored
* fixtures wrong.
+ *
+ * <p>The package-private overloads taking a resource root are shared with the
fixture families of later
+ * releases that persisted new labels ({@link Legacy415Fixtures}), which are
generated the same way.
*/
public final class Legacy413Fixtures {
@@ -56,7 +59,11 @@ public final class Legacy413Fixtures {
}
private static CatalogMgr loadCatalogMgr(String fileName) throws
IOException {
- try (InputStream in = open(fileName);
+ return loadCatalogMgr(ROOT, fileName);
+ }
+
+ static CatalogMgr loadCatalogMgr(String root, String fileName) throws
IOException {
+ try (InputStream in = open(root, fileName);
DataInputStream dis = new DataInputStream(in)) {
return CatalogMgr.read(dis);
}
@@ -64,32 +71,34 @@ public final class Legacy413Fixtures {
/** Raw bytes of one edit-log entry, in JournalEntity wire form (opCode
short + Text-framed JSON). */
public static byte[] journalEntry(String fileName) throws IOException {
- try (InputStream in = open("editlog/" + fileName)) {
- return readAll(in);
- }
+ return resource(ROOT, "editlog/" + fileName);
}
/** Names of every edit-log fixture with the given op code, e.g. {@code
320}. */
public static List<String> journalEntryNames(int opCode) {
+ return journalEntryNames(ROOT, opCode);
+ }
+
+ static List<String> journalEntryNames(String root, int opCode) {
List<String> names = new ArrayList<>();
- for (String name : listEditLogFixtures()) {
+ for (String name : listEditLogFixtures(root)) {
if (name.startsWith("op" + opCode + "-")) {
names.add(name);
}
}
if (names.isEmpty()) {
throw new IllegalStateException("no edit-log fixture for op " +
opCode
- + "; the resource index at " + ROOT + "editlog.index is
stale");
+ + "; the resource index at " + root + "editlog.index is
stale");
}
return names;
}
- private static List<String> listEditLogFixtures() {
+ private static List<String> listEditLogFixtures(String root) {
// Resource directories are not enumerable from a jar, and surefire
may well run these classes off one.
// The generator writes a flat index next to the fixtures precisely so
this does not need directory
// listing; keeping it in one place means a fixture added without
regenerating the index fails loudly
// in journalEntryNames() rather than silently reducing coverage.
- try (InputStream in = open("editlog.index")) {
+ try (InputStream in = open(root, "editlog.index")) {
String text = new String(readAll(in), StandardCharsets.UTF_8);
List<String> names = new ArrayList<>();
for (String line : text.split("\n")) {
@@ -111,8 +120,12 @@ public final class Legacy413Fixtures {
* @return label to the fully-qualified name of the class 4.1.3 mapped it
to, in file order
*/
public static Map<String, LegacyLabel> legacyLabels(String which) throws
IOException {
+ return legacyLabels(ROOT, which);
+ }
+
+ static Map<String, LegacyLabel> legacyLabels(String root, String which)
throws IOException {
Map<String, LegacyLabel> labels = new LinkedHashMap<>();
- try (InputStream in = open("labels." + which + ".txt")) {
+ try (InputStream in = open(root, "labels." + which + ".txt")) {
String text = new String(readAll(in), StandardCharsets.UTF_8);
for (String line : text.split("\n")) {
if (line.trim().isEmpty() || line.startsWith("#")) {
@@ -148,11 +161,18 @@ public final class Legacy413Fixtures {
}
}
- private static InputStream open(String relative) throws IOException {
- InputStream in = Legacy413Fixtures.class.getResourceAsStream(ROOT +
relative);
+ /** All bytes of one fixture file under the release directory {@code
root}. */
+ static byte[] resource(String root, String relative) throws IOException {
+ try (InputStream in = open(root, relative)) {
+ return readAll(in);
+ }
+ }
+
+ private static InputStream open(String root, String relative) throws
IOException {
+ InputStream in = Legacy413Fixtures.class.getResourceAsStream(root +
relative);
if (in == null) {
- throw new IOException("missing 4.1.3 fixture: " + ROOT + relative
- + " -- see src/test/resources/upgrade/413/PROVENANCE.txt");
+ throw new IOException("missing upgrade fixture: " + root + relative
+ + " -- see src/test/resources" + root + "PROVENANCE.txt");
}
return in;
}
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/datasource/Legacy415Fixtures.java
b/fe/fe-core/src/test/java/org/apache/doris/datasource/Legacy415Fixtures.java
new file mode 100644
index 00000000000..4a790acdf42
--- /dev/null
+++
b/fe/fe-core/src/test/java/org/apache/doris/datasource/Legacy415Fixtures.java
@@ -0,0 +1,74 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.datasource;
+
+import org.apache.doris.common.io.Text;
+import org.apache.doris.datasource.Legacy413Fixtures.LegacyLabel;
+
+import java.io.ByteArrayInputStream;
+import java.io.DataInputStream;
+import java.io.IOException;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * Access to the golden Doris 4.1.5 metadata fixtures under {@code
src/test/resources/upgrade/415/}: the Lance
+ * catalogs 4.1.4 introduced, in the bytes a real 4.1.5 FE writes (see the
PROVENANCE.txt next to them).
+ */
+public final class Legacy415Fixtures {
+
+ /** The lastUpdateTime the generator stamps on every catalog, so the bytes
are deterministic. */
+ public static final long FIXED_UPDATE_TIME = 1753000000000L;
+
+ private static final String ROOT = "/upgrade/415/";
+
+ private Legacy415Fixtures() {
+ }
+
+ /** The image's "datasource" module as a 4.1.5 FE would have written it. */
+ public static CatalogMgr loadCatalogMgr() throws IOException {
+ return Legacy413Fixtures.loadCatalogMgr(ROOT, "datasource.module.bin");
+ }
+
+ /** The same module's JSON payload, unframed, for assertions on what is
literally on the wire. */
+ public static String catalogMgrJson() throws IOException {
+ byte[] framed = Legacy413Fixtures.resource(ROOT,
"datasource.module.bin");
+ try (DataInputStream in = new DataInputStream(new
ByteArrayInputStream(framed))) {
+ return Text.readString(in);
+ }
+ }
+
+ /** Raw bytes of one edit-log entry, in JournalEntity wire form (opCode
short + Text-framed JSON). */
+ public static byte[] journalEntry(String fileName) throws IOException {
+ return Legacy413Fixtures.resource(ROOT, "editlog/" + fileName);
+ }
+
+ /** Names of every edit-log fixture with the given op code, e.g. {@code
320}. */
+ public static List<String> journalEntryNames(int opCode) {
+ return Legacy413Fixtures.journalEntryNames(ROOT, opCode);
+ }
+
+ /**
+ * One of 4.1.5's own GSON label registries, read out of the generated
{@code labels.*.txt}.
+ *
+ * @param which one of {@code ds}, {@code db}, {@code tbl}
+ */
+ public static Map<String, LegacyLabel> legacyLabels(String which) throws
IOException {
+ return Legacy413Fixtures.legacyLabels(ROOT, which);
+ }
+}
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/datasource/Legacy415LanceUpgradeTest.java
b/fe/fe-core/src/test/java/org/apache/doris/datasource/Legacy415LanceUpgradeTest.java
new file mode 100644
index 00000000000..096104b1dac
--- /dev/null
+++
b/fe/fe-core/src/test/java/org/apache/doris/datasource/Legacy415LanceUpgradeTest.java
@@ -0,0 +1,227 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.datasource;
+
+import org.apache.doris.common.Pair;
+import org.apache.doris.common.io.Text;
+import org.apache.doris.connector.ConnectorFactory;
+import org.apache.doris.connector.ConnectorPluginManager;
+import org.apache.doris.datasource.log.CatalogLog;
+import org.apache.doris.datasource.log.InitCatalogLog;
+import org.apache.doris.datasource.plugin.PluginDrivenExternalCatalog;
+import org.apache.doris.datasource.upgrade.RecordingConnectorProvider;
+import org.apache.doris.persist.OperationType;
+import org.apache.doris.persist.gson.GsonUtils;
+
+import com.google.common.base.Throwables;
+import com.google.common.collect.ImmutableList;
+import com.google.gson.JsonElement;
+import com.google.gson.JsonObject;
+import com.google.gson.JsonParser;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.io.ByteArrayInputStream;
+import java.io.DataInputStream;
+import java.io.IOException;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Set;
+import java.util.TreeMap;
+
+/**
+ * Upgrading a 4.1.4+ cluster that has Lance catalogs to this branch.
+ *
+ * <p>4.1.4 shipped Lance as fe-core classes ({@code LanceExternalCatalog} and
friends); this branch has no such
+ * classes and, until a Lance connector plugin lands, no provider for the
"lance" type either. The fixtures were
+ * emitted by real 4.1.5 bytecode
(src/test/resources/upgrade/415/PROVENANCE.txt). What must hold: the image
+ * loads and the FE starts, the catalog keeps everything the user configured,
it fails only when used while the
+ * plugin is missing, and a plugin that claims "lance" later is handed exactly
the persisted properties.
+ *
+ * <p>This class lives in {@code org.apache.doris.datasource} because {@link
ExternalCatalog#logType} and
+ * {@link ExternalCatalog#tableAutoAnalyzePolicy} are protected.
+ */
+public class Legacy415LanceUpgradeTest {
+
+ private static final long[] LANCE_IDS = {10101L, 10102L, 10103L};
+
+ @AfterEach
+ public void tearDown() {
+ // The plugin manager is a process-wide static shared with every other
test class in this fork.
+ ConnectorFactory.initPluginManager(new ConnectorPluginManager());
+ }
+
+ @Test
+ public void everyLanceCatalogInThe415ImageLoadsAsPluginDriven() throws
Exception {
+ // MUTATION: without the LanceExternalCatalog compatible subtype the
whole module fails to parse
+ // ("cannot deserialize ... subtype named LanceExternalCatalog"), i.e.
the upgraded FE cannot start.
+ CatalogMgr mgr = Legacy415Fixtures.loadCatalogMgr();
+
+ Assertions.assertEquals(4, mgr.getCatalogNum(),
+ "the 4.1.5 fixture holds the internal catalog plus 3 Lance
catalogs; a different count means the "
+ + "fixture was regenerated without updating this
test");
+ Assertions.assertSame(InternalCatalog.class,
mgr.getCatalog(0L).getClass());
+ for (long id : LANCE_IDS) {
+ ExternalCatalog catalog = (ExternalCatalog) mgr.getCatalog(id);
+ Assertions.assertSame(PluginDrivenExternalCatalog.class,
catalog.getClass(),
+ "catalog " + id + " must be served by the connector
framework, but loaded as "
+ + catalog.getClass().getSimpleName());
+ Assertions.assertEquals("lance", catalog.getType());
+ Assertions.assertEquals(InitCatalogLog.Type.PLUGIN,
catalog.logType,
+ "a migrated catalog must report the PLUGIN logType,
otherwise buildDbForInit takes the "
+ + "legacy path");
+ }
+ }
+
+ @Test
+ public void everyPropertyAndUserVisibleStateSurvivesTheMigration() throws
Exception {
+ CatalogMgr mgr = Legacy415Fixtures.loadCatalogMgr();
+
+ for (long id : LANCE_IDS) {
+ ExternalCatalog catalog = (ExternalCatalog) mgr.getCatalog(id);
+ Map<String, String> loaded =
catalog.getCatalogProperty().getProperties();
+ persistedProperties(id).forEach((key, value) ->
Assertions.assertEquals(value, loaded.get(key),
+ "property '" + key + "' of catalog " + id + " did not
survive loading the 4.1.5 image"));
+ }
+
+ ExternalCatalog s3 = (ExternalCatalog) mgr.getCatalog(10101L);
+ Assertions.assertEquals("g1_lance_s3", s3.getName());
+ Assertions.assertEquals("lance on s3", s3.getComment(), "the user's
COMMENT must survive the upgrade");
+ // Before anything initialises the catalog: makeSureInitialized()
overwrites lastUpdateTime.
+ Assertions.assertEquals(Legacy415Fixtures.FIXED_UPDATE_TIME,
s3.getLastUpdateTime());
+ Map<Pair<String, String>, String> policy = s3.tableAutoAnalyzePolicy;
+ Assertions.assertEquals(1, policy.size(), "auto-analyze policies must
survive the upgrade");
+ Assertions.assertEquals("enable", policy.get(Pair.of("db1", "tbl1")));
+ }
+
+ @Test
+ public void everyLogTypeNameThe415ImageCarriesStillDeserializesToItself()
throws Exception {
+ // Gson reads an enum name it does not know as null instead of
failing, so losing LANCE would not break
+ // loading: a Lance catalog always persists its "type" property and
nothing is backfilled from the
+ // logType. This pins the rule InitCatalogLog.Type documents -- every
name a released image carries
+ // stays -- for the name 4.1.4 added.
+ Set<String> names = new HashSet<>();
+ for (Map.Entry<String, JsonElement> entry :
idToCatalogJson().entrySet()) {
+ JsonElement logType =
entry.getValue().getAsJsonObject().get("logType");
+ if (logType != null) {
+ names.add(logType.getAsString());
+ }
+ }
+ Assertions.assertEquals(java.util.Collections.singleton("LANCE"),
names,
+ "the fixture's premise changed; regenerate the expectations
with the fixture");
+ for (String name : names) {
+ InitCatalogLog.Type type = GsonUtils.GSON.fromJson("\"" + name +
"\"", InitCatalogLog.Type.class);
+ Assertions.assertNotNull(type, "logType '" + name + "' written by
4.1.5 is no longer a known name");
+ Assertions.assertEquals(name, type.name());
+ }
+ }
+
+ @Test
+ public void lanceCatalogWithoutItsPluginFailsOnlyWhenUsed() throws
Exception {
+ // The state right after upgrading to a build that ships no Lance
connector: the image loaded (above),
+ // and touching the catalog reports what is missing instead of
something obscure.
+ ConnectorFactory.initPluginManager(new ConnectorPluginManager());
+ ExternalCatalog catalog = (ExternalCatalog)
Legacy415Fixtures.loadCatalogMgr().getCatalog(10102L);
+
+ RuntimeException e = Assertions.assertThrows(RuntimeException.class,
catalog::makeSureInitialized);
+ Assertions.assertTrue(Throwables.getStackTraceAsString(e).contains(
+ "No ConnectorProvider found for plugin-driven catalog:
g1_lance_local, type: lance"),
+ Throwables.getStackTraceAsString(e));
+ }
+
+ @Test
+ public void lanceConnectorIsHandedExactlyThePersistedProperties() throws
Exception {
+ // Once a plugin claims "lance", the migrated catalog must reach it
with nothing dropped or renamed --
+ // credentials and namespace options included -- or it loads fine and
then cannot connect to anything.
+ RecordingConnectorProvider.Registry registry =
+
RecordingConnectorProvider.installFor(ImmutableList.of("lance"));
+ CatalogMgr mgr = Legacy415Fixtures.loadCatalogMgr();
+
+ for (long id : LANCE_IDS) {
+ ExternalCatalog catalog = (ExternalCatalog) mgr.getCatalog(id);
+ catalog.makeSureInitialized();
+
+ RecordingConnectorProvider lance = registry.get("lance");
+ Assertions.assertEquals(catalog.getName(),
lance.calls.get(lance.calls.size() - 1).catalogName);
+ Map<String, String> delivered = lance.lastProperties();
+ persistedProperties(id).forEach((key, value) ->
Assertions.assertEquals(value, delivered.get(key),
+ "property '" + key + "' of catalog " + id + " did not
reach the connector unchanged"));
+ }
+ }
+
+ @Test
+ public void everyCreateLanceCatalogEntryFrom415ReplaysWithoutThePlugin()
throws Exception {
+ // OP_CREATE_CATALOG journals only the property map, so this path
never depended on the class label; it
+ // is pinned because replay builds the connector synchronously on the
FE's startup thread, where
+ // EditLog turns any exception into System.exit(-1). Without the
plugin the catalog registers degraded.
+ ConnectorFactory.initPluginManager(new ConnectorPluginManager());
+
+ for (String fixture :
Legacy415Fixtures.journalEntryNames(OperationType.OP_CREATE_CATALOG)) {
+ CatalogLog log = readCatalogLog(fixture);
+ CatalogIf<?> catalog = CatalogFactory.createFromLog(log);
+
+ Assertions.assertSame(PluginDrivenExternalCatalog.class,
catalog.getClass(),
+ "replay of " + fixture + " produced " +
catalog.getClass().getSimpleName());
+ Assertions.assertEquals(log.getCatalogName(), catalog.getName());
+ Assertions.assertEquals("lance", ((ExternalCatalog)
catalog).getType());
+ }
+ }
+
+ @Test
+ public void
everyCreateLanceCatalogEntryFrom415ReplaysThroughALancePlugin() throws
Exception {
+ RecordingConnectorProvider.Registry registry =
+
RecordingConnectorProvider.installFor(ImmutableList.of("lance"));
+
+ for (String fixture :
Legacy415Fixtures.journalEntryNames(OperationType.OP_CREATE_CATALOG)) {
+ CatalogLog log = readCatalogLog(fixture);
+ CatalogIf<?> catalog = CatalogFactory.createFromLog(log);
+
+ Assertions.assertSame(PluginDrivenExternalCatalog.class,
catalog.getClass());
+ RecordingConnectorProvider lance = registry.get("lance");
+ Assertions.assertEquals(log.getCatalogName(),
lance.calls.get(lance.calls.size() - 1).catalogName,
+ "the provider claiming 'lance' must be asked to build the
connector for " + fixture);
+ Map<String, String> delivered = lance.lastProperties();
+ log.getProps().forEach((key, value) ->
Assertions.assertEquals(value, delivered.get(key),
+ "property '" + key + "' did not survive the journey from
the 4.1.5 journal to the connector"));
+ }
+ }
+
+ /** The property map catalog {@code id} has on the wire, read straight out
of the 4.1.5 module's JSON. */
+ private static Map<String, String> persistedProperties(long id) throws
IOException {
+ JsonObject properties =
idToCatalogJson().getAsJsonObject(String.valueOf(id))
+
.getAsJsonObject("catalogProperty").getAsJsonObject("properties");
+ Map<String, String> result = new TreeMap<>();
+ properties.entrySet().forEach(e -> result.put(e.getKey(),
e.getValue().getAsString()));
+ Assertions.assertTrue(result.size() > 2, "catalog " + id + " has an
implausibly small property map");
+ return result;
+ }
+
+ private static JsonObject idToCatalogJson() throws IOException {
+ return
JsonParser.parseString(Legacy415Fixtures.catalogMgrJson()).getAsJsonObject()
+ .getAsJsonObject("idToCatalog");
+ }
+
+ private static CatalogLog readCatalogLog(String fixture) throws
IOException {
+ byte[] bytes = Legacy415Fixtures.journalEntry(fixture);
+ try (DataInputStream in = new DataInputStream(new
ByteArrayInputStream(bytes))) {
+ in.readShort(); // op code; the payload class is not on the wire
+ return GsonUtils.GSON.fromJson(Text.readString(in),
CatalogLog.class);
+ }
+ }
+}
diff --git a/fe/fe-core/src/test/resources/upgrade/415/PROVENANCE.txt
b/fe/fe-core/src/test/resources/upgrade/415/PROVENANCE.txt
new file mode 100644
index 00000000000..41d17d4eb1c
--- /dev/null
+++ b/fe/fe-core/src/test/resources/upgrade/415/PROVENANCE.txt
@@ -0,0 +1,56 @@
+Golden FE metadata fixtures for upgrading a 4.1.4+ cluster that has Lance
catalogs.
+
+WHAT THIS IS
+============
+Every .bin / .json / .txt / .index file in this directory except this one was
produced by REAL Apache Doris
+4.1.5 bytecode, the same way the 4.1.3 fixtures in ../413 were (read
../413/PROVENANCE.txt for why
+hand-written fixtures are not good enough). 4.1.4 added the Lance catalog;
4.1.4, 4.1.4.1, 4.1.5 and
+4.2.0-rc01 register the same GSON labels, and the only ones 4.1.3 did not have
are LanceExternalCatalog,
+LanceExternalDatabase and LanceExternalTable. 4.1.5 is the newest release at
the time of writing.
+
+PROVENANCE
+==========
+ source tag : 4.1.5-release (annotated tag; commit
917b20be83878e98f94e3ebf7efccc80346e3e50)
+ gson : 2.10.1 (identical on 4.1.5 and on the branch under test)
+ generator : generator/Gen415Fixtures.java (committed here verbatim)
+ generated on : 2026-10-24
+ sha256 datasource.module.bin :
7a435e135f73d1f2f9b9fa099dc6756e6b01529bf660ced736d923724ebb425d
+
+HOW TO REGENERATE
+=================
+ mkdir -p /path/to/doris-415 && git archive 4.1.5-release | tar -x -C
/path/to/doris-415
+ cd /path/to/doris-415
+ DORIS_HOME=$PWD DORIS_THIRDPARTY=/path/to/doris/thirdparty bash
generated-source.sh
+ # compile only, and resolve the classpath inside the same reactor so the
4.1.5 modules come from their
+ # own target/classes rather than from ~/.m2 (where 1.2-SNAPSHOT is the
CURRENT branch's):
+ mvn -o -f fe/pom.xml -pl fe-core -am -Dmaven.test.skip=true
-Dcheckstyle.skip=true -Dlicense.skip=true \
+ -Drat.skip=true -Dmaven.build.cache.enabled=false
-Dmdep.includeScope=runtime \
+ compile dependency:build-classpath > build.log
+ # the last "Dependencies classpath:" block in build.log is fe-core's;
prepend fe-core's own classes:
+ CP=$PWD/fe/fe-core/target/classes:$(grep -A1 'Dependencies classpath:'
build.log | tail -1)
+ javac -d gen-classes -cp "$CP" Gen415Fixtures.java
+ java -cp "gen-classes:$CP" org.apache.doris.tools.Gen415Fixtures <outDir>
+
+ Output is byte-deterministic across JVM runs (verified: two runs produce
identical files).
+
+ !! NEVER run `mvn install` in the 4.1.5 tree: both trees resolve
${revision}=1.2-SNAPSHOT.
+
+CONTENTS
+========
+ datasource.module.bin the image's "datasource" module, framed exactly as
MetaWriter writes it: the
+ internal catalog plus three Lance catalogs (ids
10101-10103), each built through
+ 4.1.5's own CatalogFactory.createFromLog, i.e. what
a catalog created on 4.1.4+
+ looks like. Their props always carry "type": 4.1.5
has no Lance resource type, so
+ there is no resource-backed (4.1.3 "G2") Lance shape.
+ 10101 g1_lance_s3 filesystem namespace on
S3-compatible storage, comment, taap
+ 10102 g1_lance_local filesystem namespace on
file://
+ 10103 g1_lance_rest REST namespace with bearer
auth and namespace options
+ catalogMgr.json the same JSON unframed, for human diffing. Not read
by the tests.
+ labels.{ds,db,tbl}.txt 4.1.5's own GSON label registries, read reflectively
out of GsonUtils.
+ editlog/op320-*.bin OP_CREATE_CATALOG for each Lance catalog, in
JournalEntity wire form.
+
+NOT HERE, ON PURPOSE
+====================
+ Lance databases / tables 4.1.5 never persists an external database or table
(only CatalogMgr.idToCatalog
+ is serialized), so no image or journal can carry
those two labels; they are
+ covered by labels.{db,tbl}.txt.
diff --git a/fe/fe-core/src/test/resources/upgrade/415/catalogMgr.json
b/fe/fe-core/src/test/resources/upgrade/415/catalogMgr.json
new file mode 100644
index 00000000000..e0e0b3006f3
--- /dev/null
+++ b/fe/fe-core/src/test/resources/upgrade/415/catalogMgr.json
@@ -0,0 +1 @@
+{"idToCatalog":{"0":{"clazz":"InternalCatalog"},"10101":{"clazz":"LanceExternalCatalog","id":10101,"name":"g1_lance_s3","logType":"LANCE","catalogProperty":{"properties":{"type":"lance","lance.catalog.type":"filesystem","warehouse":"s3://warehouse/lance","s3.endpoint":"http://minio:9000","s3.access_key":"ak","s3.secret_key":"sk","s3.region":"us-east-1","use_path_style":"true","use_meta_cache":"true"}},"initialized":false,"lastUpdateTime":1753000000000,"taap":[[{"first":"db1","second":"tb
[...]
\ No newline at end of file
diff --git a/fe/fe-core/src/test/resources/upgrade/415/datasource.module.bin
b/fe/fe-core/src/test/resources/upgrade/415/datasource.module.bin
new file mode 100644
index 00000000000..950442861bd
Binary files /dev/null and
b/fe/fe-core/src/test/resources/upgrade/415/datasource.module.bin differ
diff --git a/fe/fe-core/src/test/resources/upgrade/415/editlog.index
b/fe/fe-core/src/test/resources/upgrade/415/editlog.index
new file mode 100644
index 00000000000..108483b6afe
--- /dev/null
+++ b/fe/fe-core/src/test/resources/upgrade/415/editlog.index
@@ -0,0 +1,4 @@
+# generated by Gen415Fixtures -- do not edit by hand
+op320-create-g1_lance_local.bin
+op320-create-g1_lance_rest.bin
+op320-create-g1_lance_s3.bin
diff --git
a/fe/fe-core/src/test/resources/upgrade/415/editlog/op320-create-g1_lance_local.bin
b/fe/fe-core/src/test/resources/upgrade/415/editlog/op320-create-g1_lance_local.bin
new file mode 100644
index 00000000000..8b3e9d6e15e
Binary files /dev/null and
b/fe/fe-core/src/test/resources/upgrade/415/editlog/op320-create-g1_lance_local.bin
differ
diff --git
a/fe/fe-core/src/test/resources/upgrade/415/editlog/op320-create-g1_lance_rest.bin
b/fe/fe-core/src/test/resources/upgrade/415/editlog/op320-create-g1_lance_rest.bin
new file mode 100644
index 00000000000..cb7a287ec9a
Binary files /dev/null and
b/fe/fe-core/src/test/resources/upgrade/415/editlog/op320-create-g1_lance_rest.bin
differ
diff --git
a/fe/fe-core/src/test/resources/upgrade/415/editlog/op320-create-g1_lance_s3.bin
b/fe/fe-core/src/test/resources/upgrade/415/editlog/op320-create-g1_lance_s3.bin
new file mode 100644
index 00000000000..02cc6b32221
Binary files /dev/null and
b/fe/fe-core/src/test/resources/upgrade/415/editlog/op320-create-g1_lance_s3.bin
differ
diff --git
a/fe/fe-core/src/test/resources/upgrade/415/generator/Gen415Fixtures.java
b/fe/fe-core/src/test/resources/upgrade/415/generator/Gen415Fixtures.java
new file mode 100644
index 00000000000..3345ca944ca
--- /dev/null
+++ b/fe/fe-core/src/test/resources/upgrade/415/generator/Gen415Fixtures.java
@@ -0,0 +1,248 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.tools;
+
+import org.apache.doris.common.FeConstants;
+import org.apache.doris.common.io.Text;
+import org.apache.doris.datasource.CatalogFactory;
+import org.apache.doris.datasource.CatalogIf;
+import org.apache.doris.datasource.CatalogLog;
+import org.apache.doris.datasource.CatalogMgr;
+import org.apache.doris.datasource.ExternalCatalog;
+import org.apache.doris.persist.gson.GsonUtils;
+import org.apache.doris.persist.gson.RuntimeTypeAdapterFactory;
+
+import java.io.DataOutputStream;
+import java.io.File;
+import java.io.FileOutputStream;
+import java.lang.reflect.Field;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.security.MessageDigest;
+import java.util.ArrayList;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.TreeMap;
+import java.util.concurrent.ConcurrentMap;
+
+/**
+ * Generates golden metadata fixtures EXACTLY as Doris 4.1.5 writes them, for
the Lance catalogs that
+ * 4.1.4 introduced.
+ *
+ * <p>This class is NOT part of Doris. It runs only against a throwaway tree
checked out at the 4.1.5-release
+ * tag, so that the fixtures it emits are produced by real 4.1.5 bytecode
rather than by someone's idea of
+ * what 4.1.5 writes. The 4.1.3 fixtures next to these (upgrade/413) were made
the same way.</p>
+ *
+ * <p>Usage: {@code java -cp <fe-core classes + deps>
org.apache.doris.tools.Gen415Fixtures <outDir>}</p>
+ */
+public class Gen415Fixtures {
+
+ /** Fixed so the output is byte-deterministic across runs; the same value
the 4.1.3 generator stamps. */
+ private static final long FIXED_UPDATE_TIME = 1753000000000L;
+
+ private final List<CatalogIf<?>> catalogs = new ArrayList<>();
+ private final List<String> failures = new ArrayList<>();
+
+ public static void main(String[] args) throws Exception {
+ if (args.length < 1) {
+ throw new IllegalArgumentException("usage: Gen415Fixtures
<outDir>");
+ }
+ FeConstants.runningUnitTest = true;
+ new Gen415Fixtures().run(new File(args[0]));
+ }
+
+ private void run(File outDir) throws Exception {
+ if (!outDir.exists() && !outDir.mkdirs()) {
+ throw new IllegalStateException("cannot create " + outDir);
+ }
+ buildLance();
+
+ CatalogMgr mgr = newCatalogMgrWith(catalogs);
+ String json = GsonUtils.GSON.toJson(mgr);
+
+ writeFramed(new File(outDir, "datasource.module.bin"), json);
+ Files.write(Paths.get(new File(outDir, "catalogMgr.json").toURI()),
json.getBytes(StandardCharsets.UTF_8));
+ dumpLabels(outDir);
+ buildEditLogFixtures(new File(outDir, "editlog"));
+
+ System.out.println("=== catalogs written: " + catalogs.size());
+ for (CatalogIf<?> c : catalogs) {
+ System.out.println(" " + c.getId() + " " +
c.getClass().getSimpleName() + " " + c.getName());
+ }
+ System.out.println("=== module sha256: " +
sha256(json.getBytes(StandardCharsets.UTF_8)));
+ if (!failures.isEmpty()) {
+ System.out.println("=== FAILURES (" + failures.size() + "):");
+ failures.forEach(f -> System.out.println(" " + f));
+ throw new IllegalStateException(failures.size() + " fixture(s)
could not be built");
+ }
+ }
+
+ //
---------------------------------------------------------------------------------------------------
+ // Lance catalogs, built through 4.1.5's own CatalogFactory replay path,
i.e. what a catalog CREATED ON
+ // 4.1.4+ looks like. Their props always carry "type":
Resource.ResourceType has no LANCE value, so a
+ // resource-backed Lance catalog cannot exist and there is no 4.1.3-style
G2 family to add here.
+ //
---------------------------------------------------------------------------------------------------
+ private void buildLance() {
+ // The shape of the external_table_p0/lance suites: a filesystem
namespace on S3-compatible storage.
+ lance(10101, "g1_lance_s3", "lance on s3", props("type", "lance",
+ "lance.catalog.type", "filesystem", "warehouse",
"s3://warehouse/lance",
+ "s3.endpoint", "http://minio:9000";, "s3.access_key", "ak",
"s3.secret_key", "sk",
+ "s3.region", "us-east-1", "use_path_style", "true"), true);
+ lance(10102, "g1_lance_local", "", props("type", "lance",
+ "lance.catalog.type", "filesystem", "warehouse",
"file:///tmp/lance_wh"), false);
+ lance(10103, "g1_lance_rest", "", props("type", "lance",
+ "lance.catalog.type", "rest", "lance.rest.uri",
"http://lance-rest:2333";,
+ "lance.rest.security.type", "bearer",
"lance.rest.bearer-token", "tok",
+ "lance.namespace.delimiter", ".",
"lance.namespace.root_database", "default"), false);
+ }
+
+ private void lance(long id, String name, String comment, Map<String,
String> props, boolean withTaap) {
+ try {
+ CatalogLog log = new CatalogLog();
+ log.setCatalogId(id);
+ log.setCatalogName(name);
+ log.setComment(comment);
+ log.setProps(props);
+ CatalogIf<?> c = CatalogFactory.createFromLog(log);
+ decorate(c, withTaap);
+ catalogs.add(c);
+ } catch (Throwable t) {
+ failures.add(name + ": " + t);
+ }
+ }
+
+ private void decorate(CatalogIf<?> c, boolean withTaap) {
+ ExternalCatalog ec = (ExternalCatalog) c;
+ ec.setLastUpdateTime(FIXED_UPDATE_TIME);
+ if (withTaap) {
+ // Empty map serializes as {}, non-empty as an array of [key,
value] 2-tuples
+ // (enableComplexMapKeySerialization); keep one catalog on each
encoding.
+ ec.setAutoAnalyzePolicy("db1", "tbl1", "enable");
+ }
+ }
+
+ //
---------------------------------------------------------------------------------------------------
+ // Edit-log fixtures: JournalEntity wire form is writeShort(opCode) +
payload.write(out), and
+ // CatalogLog.write is Text.writeString(out, GSON.toJson(this)).
OP_CREATE_CATALOG only; the other
+ // catalog ops carry no catalog type and are covered by the 4.1.3 fixtures.
+ //
---------------------------------------------------------------------------------------------------
+ private void buildEditLogFixtures(File dir) throws Exception {
+ if (!dir.exists() && !dir.mkdirs()) {
+ throw new IllegalStateException("cannot create " + dir);
+ }
+ for (CatalogIf<?> c : catalogs) {
+ writeOp(dir, 320, "create-" + c.getName(), c.constructEditLog());
+ }
+
+ String[] names = dir.list();
+ if (names == null) {
+ throw new IllegalStateException("cannot list " + dir);
+ }
+ java.util.Arrays.sort(names);
+ StringBuilder sb = new StringBuilder("# generated by Gen415Fixtures --
do not edit by hand\n");
+ for (String n : names) {
+ sb.append(n).append('\n');
+ }
+ Files.write(Paths.get(new File(dir.getParentFile(),
"editlog.index").toURI()),
+ sb.toString().getBytes(StandardCharsets.UTF_8));
+ }
+
+ private void writeOp(File dir, int opCode, String name, CatalogLog log)
throws Exception {
+ File f = new File(dir, String.format("op%d-%s.bin", opCode, name));
+ try (DataOutputStream out = new DataOutputStream(new
FileOutputStream(f))) {
+ out.writeShort(opCode);
+ log.write(out);
+ }
+ }
+
+ //
---------------------------------------------------------------------------------------------------
+ // Label inventory, read reflectively out of 4.1.5's own registries (a
regex scrape drops labels whose
+ // class name contains a digit).
+ //
---------------------------------------------------------------------------------------------------
+ private void dumpLabels(File outDir) throws Exception {
+ dumpOneRegistry(outDir, "labels.ds.txt", "dsTypeAdapterFactory");
+ dumpOneRegistry(outDir, "labels.db.txt", "dbTypeAdapterFactory");
+ dumpOneRegistry(outDir, "labels.tbl.txt", "tblTypeAdapterFactory");
+ }
+
+ private void dumpOneRegistry(File outDir, String fileName, String
fieldName) throws Exception {
+ Field f = GsonUtils.class.getDeclaredField(fieldName);
+ f.setAccessible(true);
+ RuntimeTypeAdapterFactory<?> factory = (RuntimeTypeAdapterFactory<?>)
f.get(null);
+ Field l =
RuntimeTypeAdapterFactory.class.getDeclaredField("labelToSubtype");
+ l.setAccessible(true);
+ @SuppressWarnings("unchecked")
+ Map<String, Class<?>> labelToSubtype = (Map<String, Class<?>>)
l.get(factory);
+ StringBuilder sb = new StringBuilder();
+ sb.append("# generated from 4.1.5 by Gen415Fixtures -- do not edit by
hand\n");
+ sb.append("# <label>\\t<target class FQN>\\t<ABSTRACT if the label can
never be emitted>\n");
+ new TreeMap<>(labelToSubtype).forEach((label, clazz) ->
sb.append(label).append('\t')
+ .append(clazz.getName())
+
.append(java.lang.reflect.Modifier.isAbstract(clazz.getModifiers()) ?
"\tABSTRACT" : "")
+ .append('\n'));
+ Files.write(Paths.get(new File(outDir, fileName).toURI()),
sb.toString().getBytes(StandardCharsets.UTF_8));
+ }
+
+ //
---------------------------------------------------------------------------------------------------
+ // helpers (same as the 4.1.3 generator)
+ //
---------------------------------------------------------------------------------------------------
+
+ /**
+ * A CatalogMgr holding the internal catalog (added by its own ctor) plus
the given externals, injected
+ * straight into idToCatalog: addCatalog() needs a live Env. Only
idToCatalog is @SerializedName.
+ */
+ private CatalogMgr newCatalogMgrWith(List<CatalogIf<?>> externals) throws
Exception {
+ CatalogMgr mgr = new CatalogMgr();
+ Field f = CatalogMgr.class.getDeclaredField("idToCatalog");
+ f.setAccessible(true);
+ @SuppressWarnings("unchecked")
+ ConcurrentMap<Long, CatalogIf<?>> idToCatalog = (ConcurrentMap<Long,
CatalogIf<?>>) f.get(mgr);
+ for (CatalogIf<?> c : externals) {
+ idToCatalog.put(c.getId(), c);
+ }
+ return mgr;
+ }
+
+ private static void writeFramed(File file, String json) throws Exception {
+ try (DataOutputStream out = new DataOutputStream(new
FileOutputStream(file))) {
+ Text.writeString(out, json);
+ }
+ }
+
+ private static Map<String, String> props(String... kv) {
+ if (kv.length % 2 != 0) {
+ throw new IllegalArgumentException("odd number of props");
+ }
+ Map<String, String> m = new LinkedHashMap<>();
+ for (int i = 0; i < kv.length; i += 2) {
+ m.put(kv[i], kv[i + 1]);
+ }
+ return m;
+ }
+
+ private static String sha256(byte[] bytes) throws Exception {
+ MessageDigest md = MessageDigest.getInstance("SHA-256");
+ StringBuilder sb = new StringBuilder();
+ for (byte b : md.digest(bytes)) {
+ sb.append(String.format("%02x", b));
+ }
+ return sb.toString();
+ }
+}
diff --git a/fe/fe-core/src/test/resources/upgrade/415/labels.db.txt
b/fe/fe-core/src/test/resources/upgrade/415/labels.db.txt
new file mode 100644
index 00000000000..7140ed51acd
--- /dev/null
+++ b/fe/fe-core/src/test/resources/upgrade/415/labels.db.txt
@@ -0,0 +1,15 @@
+# generated from 4.1.5 by Gen415Fixtures -- do not edit by hand
+# <label>\t<target class FQN>\t<ABSTRACT if the label can never be emitted>
+EsExternalDatabase org.apache.doris.datasource.es.EsExternalDatabase
+ExternalDatabase org.apache.doris.datasource.ExternalDatabase ABSTRACT
+ExternalInfoSchemaDatabase
org.apache.doris.datasource.infoschema.ExternalInfoSchemaDatabase
+ExternalMysqlDatabase
org.apache.doris.datasource.infoschema.ExternalMysqlDatabase
+HMSExternalDatabase org.apache.doris.datasource.hive.HMSExternalDatabase
+IcebergExternalDatabase
org.apache.doris.datasource.iceberg.IcebergExternalDatabase
+JdbcExternalDatabase org.apache.doris.datasource.jdbc.JdbcExternalDatabase
+LakeSoulExternalDatabase
org.apache.doris.datasource.lakesoul.LakeSoulExternalDatabase
+LanceExternalDatabase org.apache.doris.datasource.lance.LanceExternalDatabase
+MaxComputeExternalDatabase
org.apache.doris.datasource.maxcompute.MaxComputeExternalDatabase
+PaimonExternalDatabase
org.apache.doris.datasource.paimon.PaimonExternalDatabase
+TestExternalDatabase org.apache.doris.datasource.test.TestExternalDatabase
+TrinoConnectorExternalDatabase
org.apache.doris.datasource.trinoconnector.TrinoConnectorExternalDatabase
diff --git a/fe/fe-core/src/test/resources/upgrade/415/labels.ds.txt
b/fe/fe-core/src/test/resources/upgrade/415/labels.ds.txt
new file mode 100644
index 00000000000..6fef4b8a617
--- /dev/null
+++ b/fe/fe-core/src/test/resources/upgrade/415/labels.ds.txt
@@ -0,0 +1,26 @@
+# generated from 4.1.5 by Gen415Fixtures -- do not edit by hand
+# <label>\t<target class FQN>\t<ABSTRACT if the label can never be emitted>
+CloudInternalCatalog org.apache.doris.cloud.datasource.CloudInternalCatalog
+EsExternalCatalog org.apache.doris.datasource.es.EsExternalCatalog
+HMSExternalCatalog org.apache.doris.datasource.hive.HMSExternalCatalog
+IcebergDLFExternalCatalog
org.apache.doris.datasource.iceberg.IcebergDLFExternalCatalog
+IcebergExternalCatalog
org.apache.doris.datasource.iceberg.IcebergExternalCatalog ABSTRACT
+IcebergGlueExternalCatalog
org.apache.doris.datasource.iceberg.IcebergGlueExternalCatalog
+IcebergHMSExternalCatalog
org.apache.doris.datasource.iceberg.IcebergHMSExternalCatalog
+IcebergHadoopExternalCatalog
org.apache.doris.datasource.iceberg.IcebergHadoopExternalCatalog
+IcebergJdbcExternalCatalog
org.apache.doris.datasource.iceberg.IcebergJdbcExternalCatalog
+IcebergRestExternalCatalog
org.apache.doris.datasource.iceberg.IcebergRestExternalCatalog
+IcebergS3TablesExternalCatalog
org.apache.doris.datasource.iceberg.IcebergS3TablesExternalCatalog
+InternalCatalog org.apache.doris.datasource.InternalCatalog
+JdbcExternalCatalog org.apache.doris.datasource.jdbc.JdbcExternalCatalog
+LakeSoulExternalCatalog
org.apache.doris.datasource.lakesoul.LakeSoulExternalCatalog
+LanceExternalCatalog org.apache.doris.datasource.lance.LanceExternalCatalog
+MaxComputeExternalCatalog
org.apache.doris.datasource.maxcompute.MaxComputeExternalCatalog
+PaimonDLFExternalCatalog
org.apache.doris.datasource.paimon.PaimonDLFExternalCatalog
+PaimonExternalCatalog org.apache.doris.datasource.paimon.PaimonExternalCatalog
+PaimonFileExternalCatalog
org.apache.doris.datasource.paimon.PaimonFileExternalCatalog
+PaimonHMSExternalCatalog
org.apache.doris.datasource.paimon.PaimonHMSExternalCatalog
+PaimonRestExternalCatalog
org.apache.doris.datasource.paimon.PaimonRestExternalCatalog
+RemoteDorisExternalCatalog
org.apache.doris.datasource.doris.RemoteDorisExternalCatalog
+TestExternalCatalog org.apache.doris.datasource.test.TestExternalCatalog
+TrinoConnectorExternalCatalog
org.apache.doris.datasource.trinoconnector.TrinoConnectorExternalCatalog
diff --git a/fe/fe-core/src/test/resources/upgrade/415/labels.tbl.txt
b/fe/fe-core/src/test/resources/upgrade/415/labels.tbl.txt
new file mode 100644
index 00000000000..43f20a73d36
--- /dev/null
+++ b/fe/fe-core/src/test/resources/upgrade/415/labels.tbl.txt
@@ -0,0 +1,29 @@
+# generated from 4.1.5 by Gen415Fixtures -- do not edit by hand
+# <label>\t<target class FQN>\t<ABSTRACT if the label can never be emitted>
+BrokerTable org.apache.doris.catalog.BrokerTable
+Dictionary org.apache.doris.dictionary.Dictionary
+EsExternalTable org.apache.doris.datasource.es.EsExternalTable
+EsTable org.apache.doris.catalog.EsTable
+ExternalInfoSchemaTable
org.apache.doris.datasource.infoschema.ExternalInfoSchemaTable
+ExternalMysqlTable
org.apache.doris.datasource.infoschema.ExternalMysqlTable
+ExternalTable org.apache.doris.datasource.ExternalTable
+FunctionGenTable org.apache.doris.catalog.FunctionGenTable
+HMSExternalTable org.apache.doris.datasource.hive.HMSExternalTable
+HiveTable org.apache.doris.catalog.HiveTable
+IcebergExternalTable org.apache.doris.datasource.iceberg.IcebergExternalTable
+InlineView org.apache.doris.catalog.InlineView
+JdbcExternalTable org.apache.doris.datasource.jdbc.JdbcExternalTable
+JdbcTable org.apache.doris.catalog.JdbcTable
+LakeSoulExternalTable
org.apache.doris.datasource.lakesoul.LakeSoulExternalTable
+LanceExternalTable org.apache.doris.datasource.lance.LanceExternalTable
+MTMV org.apache.doris.catalog.MTMV
+MaxComputeExternalTable
org.apache.doris.datasource.maxcompute.MaxComputeExternalTable
+MysqlDBTable org.apache.doris.catalog.MysqlDBTable
+MysqlTable org.apache.doris.catalog.MysqlTable
+OdbcTable org.apache.doris.catalog.OdbcTable
+OlapTable org.apache.doris.catalog.OlapTable
+PaimonExternalTable org.apache.doris.datasource.paimon.PaimonExternalTable
+SchemaTable org.apache.doris.catalog.SchemaTable
+TestExternalTable org.apache.doris.datasource.test.TestExternalTable
+TrinoConnectorExternalTable
org.apache.doris.datasource.trinoconnector.TrinoConnectorExternalTable
+View org.apache.doris.catalog.View
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
