SEPURI-SAI-KRISHNA opened a new issue, #13038:
URL: https://github.com/apache/seatunnel/issues/13038
### Search before asking
- [X] I had searched in the issues and found no similar issues.
### What happened
Databend has no TIME type, but the Databend connector names one in the DDL
it generates for a
schema change event, so the statement cannot be parsed and the schema change
fails.
**Databend has no TIME type.** Its type parser defines 29 column type rules,
and none of them is a
standalone TIME. The token appears only inside `TIMESTAMP WITH TIME ZONE`
(`src/query/ast/src/parser/expr.rs`, the `ty_*` bindings and
`ty_timestamp_tz`):
```rust
let ty_date = value(TypeName::Date, rule! { DATE });
let ty_datetime = map(rule! { ( DATETIME | TIMESTAMP ) ~ ( "(" ~
^#literal_u64 ~ ^")" )? }, ...);
let ty_timestamp_tz = value(TypeName::TimestampTz, rule! { TIMESTAMP ~ WITH
~ TIME ~ ZONE });
```
There is no `TableDataType::Time` variant either, so nothing in Databend can
hold or report one.
**The connector still emits it.** `DatabendTypeConverter.mapToDatabendType`:
```java
case TIME:
return "TIME";
```
and `SchemaChangeManager.applyAddColumn` appends that straight into the
statement:
```java
BasicTypeDefine typeDefine =
DatabendTypeConverter.convertToDatabendType(event.getColumn());
StringBuilder sqlBuilder =
new StringBuilder()
.append("ALTER TABLE
").append(quoteIdentifier(tablePath.getFullName()))
.append(" ADD COLUMN
").append(quoteIdentifier(event.getColumn().getName()))
.append(" ").append(typeDefine.getColumnType());
```
producing
```sql
ALTER TABLE `default`.`orders` ADD COLUMN `started_at` TIME
```
which Databend cannot parse. `applyModifyColumn` builds its statement the
same way.
**The path is reachable, and a common setup reaches it.** A MySQL TIME
column becomes a SeaTunnel
TIME: `MySqlTypeConverter` maps `case MYSQL_TIME:
builder.dataType(LocalTimeType.LOCAL_TIME_TYPE)`.
So with a MySQL CDC source writing to a Databend sink and
`enable_schema_evolution = true`, an
`ALTER TABLE ... ADD COLUMN t TIME` on the MySQL side produces an
`AlterTableAddColumnEvent`
carrying a TIME column, and the chain is
```
DatabendSinkWriter.applySchemaChange (SupportSchemaEvolutionSinkWriter)
-> SchemaChangeManager.applySchemaChange
-> SchemaChangeManager.applyAddColumn
-> DatabendTypeConverter.convertToDatabendType -> "TIME"
```
**The documentation claims the mapping exists, in four places.** The source
tables
(`docs/en/connectors/source/Databend.md`,
`docs/zh/connectors/source/Databend.md`) list
`TIME | TIME`, but no Databend column is ever a TIME and
`DatabendCatalog.convertDatabendType` has
no `case "TIME"` to read one with, so that row describes nothing. The sink
tables
(`docs/en/connectors/sink/Databend.md`,
`docs/zh/connectors/sink/Databend.md`) also list
`TIME | TIME`, which is the mapping this issue is about.
**A second code site has the same defect.**
`DatabendCatalog.toDatabendTypeString` also returns
`"TIME"`, on the `Catalog.createTable` path used by `schema_save_mode`. I am
raising this issue and
a PR for the `DatabendTypeConverter` site, and will follow up for the
catalog site separately,
because that file is being changed in #13023 and its unit test does not
exist on `dev` yet.
For reference, the connector's own sink writer already chose not to emit
TIME: its (currently
unreachable) mapper returns `TIMESTAMP` for the same type. Doris and
StarRocks, which also have no
TIME type, both carry it as text.
### SeaTunnel Version
dev, verified at commit `403efeee0`.
### SeaTunnel Config
```conf
env {
job.mode = "STREAMING"
}
source {
MySQL-CDC {
base-url = "jdbc:mysql://mysql:3306/shop"
username = "root"
password = "root"
table-names = ["shop.orders"]
}
}
sink {
Databend {
url = "jdbc:databend://databend:8000/default"
username = "root"
password = ""
database = "default"
table = "orders"
schema_save_mode = "CREATE_SCHEMA_WHEN_NOT_EXIST"
enable_schema_evolution = true
}
}
```
With the job running, on the MySQL side:
```sql
ALTER TABLE shop.orders ADD COLUMN started_at TIME;
```
### Running Command
```shell
./bin/seatunnel.sh --config config/mysql_cdc_to_databend.conf -e local
```
### Error Exception
```log
No server log is quoted here on purpose. I verified this from the code and
from Databend's type
parser rather than from a run, so quoting an exception would mean inventing
one. Docker is not
available in my environment, and no existing E2E in connector-databend-e2e
drives a schema change
event, so I could not capture the real message.
What is certain is the statement that gets built and that Databend has no
parser rule that accepts
it:
ALTER TABLE `default`.`orders` ADD COLUMN `started_at` TIME
```
### Zeta or Flink or Spark Version
Not version specific. The statement is built in the sink writer, before any
engine detail applies.
### Java or Scala Version
Not version specific.
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's Code of Conduct
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
