corgy-w opened a new pull request, #13305:
URL: https://github.com/apache/seatunnel/pull/13305
## What
`MySqlDdlBuilder.generateDdl()` always appended a `PRIMARY KEY (...)` clause
built with
`Collectors.joining(", ", "PRIMARY KEY (", ")")`:
```java
String keyDefinitions =
primaryKeys.stream()
.map(MySqlUtils::quote)
.collect(Collectors.joining(", ", "PRIMARY KEY (", ")"));
return String.format(
"CREATE TABLE %s (%s, %s)", tableId.table(), columnDefinitions,
keyDefinitions);
```
For a table without a primary key the stream is empty, and
`Collectors.joining` still returns the
prefix and the suffix, so the generated statement became:
```sql
CREATE TABLE no_pk_table (`id` int NOT NULL, `name` varchar(64) , PRIMARY
KEY ())
```
## Why
That DDL is fed to Debezium's DDL parser by
`MySqlSchema.getTableSchemaByDescTable()`, the fallback
used when `SHOW CREATE TABLE` fails. The parser rejects the statement, so no
schema change event is
produced and the table schema is lost (in a job this surfaces as `Can't
obtain schema for table
<table>`):
```
WARN io.debezium.connector.mysql.MySqlDatabaseSchema - Ignoring unparseable
DDL statement
'CREATE TABLE no_pk_table (`id` int NOT NULL, `name` varchar(64) , PRIMARY
KEY ())':
io.debezium.text.ParsingException: DDL statement couldn't be parsed.
```
Loading the schema of a primary-key-less table through the `DESC` path was
therefore impossible.
## Fix
Emit the `PRIMARY KEY` clause only when there is at least one primary key
column. The quoting and
the comma placement of the non-empty case are unchanged:
```java
public String generateDdl() {
String columnDefinitions =
columns.stream().map(Column::generateDdl).collect(Collectors.joining(", "));
StringBuilder ddl =
new StringBuilder(
String.format("CREATE TABLE %s (%s", tableId.table(),
columnDefinitions));
// A table without a primary key has no key columns, and
Collectors.joining would still
// return the prefix/suffix ("PRIMARY KEY ()") for an empty stream.
Emitting that clause
// makes the DDL unparseable, which breaks the `DESC` fallback in
MySqlSchema.
if (!primaryKeys.isEmpty()) {
ddl.append(", ")
.append(
primaryKeys.stream()
.map(MySqlUtils::quote)
.collect(Collectors.joining(", ", "PRIMARY
KEY (", ")")));
}
return ddl.append(")").toString();
}
```
`columnDefinitions` is the only other clause in this method; it cannot be
empty (a MySQL table
always has at least one column, so `DESC` always returns at least one row),
so it is left as is.
## Test
New integration test `MysqlSchemaDescIT` under
`seatunnel-e2e/seatunnel-connector-v2-e2e/connector-cdc-mysql-e2e` — no unit
test is added or
modified. It is a standalone JUnit 5 test (it does not extend
`TestSuiteBase`, so it needs no
SeaTunnel distribution image): it starts a real `MySqlContainer` (MySQL
8.0.43, same my.cnf/setup
wiring as `MysqlCDCIT`), creates
```sql
CREATE TABLE `mysql_schema_desc`.`no_pk_table` (`id` INT NOT NULL, `name`
VARCHAR(64));
CREATE TABLE `mysql_schema_desc`.`pk_table` (`id` INT NOT NULL, `name`
VARCHAR(64), PRIMARY KEY (`id`));
```
builds a real `MySqlSourceConfig` (`MySqlSourceConfigFactory`) and a real
Debezium `JdbcConnection`
(`MySqlConnectionUtils.createMySqlConnection(...).connect()`), constructs
`MySqlSchema` and calls
the `@VisibleForTesting MySqlSchema#readTableSchemaByDesc(jdbc, tableId)`
for both tables.
Assertions: both tables load (no exception, non-null `TableChange` of type
`CREATE`, columns `id`
and `name` present, `INT`/`VARCHAR` type names); `primaryKeyColumnNames()`
is empty for
`no_pk_table` and `["id"]` for `pk_table`.
### How it was run
```
./mvnw -o -pl
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-cdc-mysql-e2e test-compile
./mvnw -o -pl
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-cdc-mysql-e2e verify \
-Dit.test=MysqlSchemaDescIT -DskipIT=false -DskipUT=true
```
Red/green against the real container:
* before the fix: `Tests run: 2, Failures: 1` —
`testDescTableWithoutPrimaryKey` fails with
`no table change was produced for mysql_schema_desc.no_pk_table`, and the
MySQL container logs the
`Ignoring unparseable DDL statement ... PRIMARY KEY ()` warning quoted
above; the `pk_table`
control case passes.
* after the fix: `Tests run: 2, Failures: 0, Errors: 0, Skipped: 0` / `BUILD
SUCCESS`.
The existing unit tests of the module still pass (`Tests run: 59, Failures:
0, Errors: 0`),
including `MySqlSchemaTest`.
## Compatibility
None: the change only drops a clause that the parser could never accept.
Tables with a primary key
generate exactly the same DDL as before.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]