SEPURI-SAI-KRISHNA opened a new pull request, #13190:
URL: https://github.com/apache/seatunnel/pull/13190

   Yes, glad to contribute to Apache SeaTunnel.
   
   ### Purpose of this pull request
   
   Closes #13188. Follow-up to #13040, as promised there (Issue 1 of its 
review).
   
   A Databend sink job with a `TIME` column failed in two places, and both are 
fixed here:
   
   - `DatabendCatalog.toDatabendTypeString` returned `TIME`, which Databend 
does not have, so creating the table through `schema_save_mode` failed. It now 
returns `STRING`, the type `DatabendTypeConverter` has used for schema changes 
since #13040. The method is now static and package private, so the unit test 
can call it.
   - Outside CDC mode, `DatabendSinkWriter.processRow` passed every value to 
`setObject`, and databend-jdbc 0.3.7 throws `Unsupported object type: 
java.time.LocalTime`. A `LocalTime` is now written with `setString`, formatted 
with `DateTimeFormatter.ISO_LOCAL_TIME`, so it always has seconds (`08:00:00`) 
and keeps a fraction of a second (`10:15:30.123`). `LocalTime.toString()` would 
give `08:00`.
   
   CDC mode is not changed: it does not call `setObject` for row values, and 
writes each row as JSON text instead.
   
   `DatabendSink.convertToDatabendType` and 
`DatabendSinkWriter.convertToDatabendType` still map `TIME` to `TIMESTAMP`, but 
nothing reaches them: the first has no caller, and the second is only called 
from `createTable` and `verifyTableSchema`, which have no callers. Tables are 
created through the catalog. I left them alone to keep this change small.
   
   The sink docs row becomes `TIME | STRING`, in EN and ZH. #13072 makes the 
same change to those two lines. I merged both ways with `git merge-file`, and 
there is no conflict in either order.
   
   Not in this PR: `TIMESTAMP` columns hit the same driver error outside CDC 
mode (`Unsupported object type: java.time.LocalDateTime`). That is a separate 
problem, and I will raise it separately.
   
   ### Does this PR introduce _any_ user-facing change?
   
   Yes. A `TIME` column is now created as `STRING` in Databend, and its values 
are written as text. Before, such a job failed, so no working job changes.
   
   ### How was this patch tested?
   
   - `DatabendCatalogTest.aTimeColumnIsCreatedAsString`: the catalog names 
`STRING`, the same type `DatabendTypeConverter` gives.
   - `DatabendSinkWriterTest.testProcessRowWritesTimeAsText`: a row with an 
`INT` and two `TIME` values binds `1` with `setObject`, and `08:00:00` and 
`10:15:30.123` with `setString`.
   - With each fix reverted, its test fails (`expected: <STRING> but was: 
<TIME>`, and the `setString` check).
   - All 25 `connector-databend` unit tests pass on JDK 17 and with the test 
JVM on JDK 11. `spotless:check` passes.
   - E2E: `DatabendIT.testFakeTimeToDatabend` runs FakeSource with a `TIME` 
column into a table the sink creates, on every engine, and reads back 
`08:00:00` and `10:15:30.123`. I have no Docker here, so I could not run it 
locally. The fork run below is its check.
   
   I also ran the real driver's `setObject`, and FakeSource's time parser on 
the two strings the E2E uses.
   
   Fork run: 
https://github.com/SEPURI-SAI-KRISHNA/seatunnel/actions/runs/42402787550 is 
green. `DatabendIT` passes 57 of 57 tests on JDK 11 and 17, and 
`testFakeTimeToDatabend` ran on Zeta, Flink 1.13, 1.15, 1.18 and 1.20, and 
Spark 2.4 and 3.3.
   
   ### Check list
   
   * [X] If any new Jar binary package adding in your PR, please add License 
Notice according
     [New License 
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
 (no new jar)
   * [X] If necessary, please update the documentation to describe the new 
feature. https://github.com/apache/seatunnel/tree/dev/docs: the `TIME` row of 
the sink docs, EN and ZH.
   * [X] If necessary, please update `incompatible-changes.md` to describe the 
incompatibility caused by this PR: not needed, the changed cases failed before.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

  • [PR] [Fix][Connector-V2][Datab... SEPURI-SAI-KRISHNA (via GitHub) via commits

Reply via email to