LeonYoah opened a new pull request, #13278:
URL: https://github.com/apache/seatunnel/pull/13278

   This change contributes to Seatunnel.
   
   ### Purpose of this pull request
   
   Close #10512.
   
   Oracle CDC does not deliver CLOB, NCLOB, BLOB, or XMLTYPE. Snapshot JDBC 
dropped those values. Incremental events either skipped them or, with 
`debezium.lob.enabled = true`, wrote `__debezium_unavailable_value` into the 
sink when the statement did not change the LOB column.
   
   This PR captures those types on the Oracle CDC source (snapshot + LogMiner). 
LogMiner LOB/XML handling is ported from Debezium 3.7.1.Final onto SeaTunnel’s 
Debezium 1.9.8 / Java 8 overlay. Overlay types keep the upstream names; 
SeaTunnel-only deltas are tagged `[SeaTunnel]`.
   
   - Snapshot reads CLOB / NCLOB / BLOB / XMLTYPE through JDBC.
   - Incremental events mine LOB/XML redo when `lob.enabled = true`.
   - An unchanged LOB on UPDATE, and LOB columns on DELETE / UPDATE_BEFORE, 
stay the unavailable placeholder unless `lob.reselect.enabled = true` looks the 
current value up by primary key (`AS OF SCN`).
   - `EMPTY_CLOB()` / `EMPTY_BLOB()` become empty values. SQL `NULL` stays null.
   
   This change does not modify the Oracle JDBC sink. `CREATE TABLE ... OF 
XMLTYPE` object tables are not supported.
   
   ### Does this PR introduce _any_ user-facing change?
   
   Yes. New Oracle-CDC options (defaults keep previous behavior):
   
   | Option | Default | Meaning |
   |---|---|---|
   | `lob.enabled` | false (omit leaves `debezium.lob.enabled` unchanged) | 
Capture CLOB, NCLOB, BLOB, and XMLTYPE in incremental events |
   | `lob.reselect.enabled` | false | On INSERT / UPDATE_AFTER, replace the 
unavailable placeholder with a primary-key lookup. Requires `lob.enabled = 
true` and a primary key |
   | `lob.unavailable-value.handling` | `warn_and_keep` | When a placeholder 
remains: `warn_and_keep` / `null` / `fail`. `null` and `fail` require 
`lob.enabled = true` |
   
   XMLTYPE is STRING. Copy `xdb` and `xmlparserv2` (same version as `ojdbc8`) 
next to the driver. Docs: `docs/en/connectors/source/Oracle-CDC.md` and 
`docs/zh/connectors/source/Oracle-CDC.md`.
   
   If XML parsing fails after those jars are added, start the engine with 
`-Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl`.
   
   ### How was this patch tested?
   
   Unit tests: `./mvnw -pl :connector-cdc-oracle test` → **105 run, 0 fail, 1 
skip** (`OracleJdbcLobXeTest` needs `-Doracle.xe.it=true`). `./mvnw -pl 
:connector-cdc-base -Dtest=IncrementalSourceReaderTest test` → **19 run, 0 
fail**.
   
   19c LogMiner E2E used `seatunnelhub/oracle-19.3.0-ee:non-cdb` (Zeta), jobs 
`oraclecdc_to_oracle_lob.conf` / `oraclecdc_to_oracle_xml.conf` (`lob.enabled = 
true`, `lob.reselect.enabled = true`, `lob.unavailable-value.handling = 
"null"`). Source vs sink compared until equal.
   
   | Scenario | Phase | Type | Environment | Result |
   |---|---|---|---|---|
   | Snapshot CLOB / NCLOB / BLOB | snapshot | CLOB, NCLOB, BLOB | 19c EE E2E | 
Pass |
   | Varchar-only UPDATE (placeholder re-select) | streaming | CLOB / XMLTYPE | 
19c EE E2E | Pass |
   | Streaming INSERT | streaming | CLOB, NCLOB, BLOB, XMLTYPE | 19c EE E2E | 
Pass |
   | Out-of-line CLOB (~5KB, `DBMS_LOB.WRITEAPPEND`) | streaming | CLOB | 19c 
EE E2E | Pass |
   | Emoji in CLOB / NCLOB | streaming | CLOB, NCLOB | 19c EE E2E | Pass |
   | Apostrophe in CLOB / NCLOB | streaming | CLOB, NCLOB | 19c EE E2E | Pass |
   | PNG BLOB | streaming | BLOB | 19c EE E2E | Pass |
   | SQL NULL CLOB | streaming | CLOB | 19c EE E2E | Pass |
   | `EMPTY_CLOB()` / `EMPTY_BLOB()` | streaming | CLOB, BLOB | 19c EE E2E | 
Pass |
   | DELETE | streaming | row delete | 19c EE E2E | Pass |
   | Snapshot XMLTYPE as STRING | snapshot | XMLTYPE | 19c EE E2E | Pass |
   | Streaming XML insert / update | streaming | XMLTYPE | 19c EE E2E | Pass |
   | Apostrophe in XML | streaming | XMLTYPE | 19c EE E2E | Pass |
   | NULL XMLTYPE | streaming | XMLTYPE | 19c EE E2E | Pass |
   | ~8KB XML (XML_BEGIN / XML_WRITE / XML_END) | streaming | XMLTYPE | 19c EE 
E2E | Pass |
   | XML_BEGIN / XML_WRITE assembly | unit | XMLTYPE redo | 
`XmlBeginParserTest`, `XmlWriteAssemblyTest` | Pass |
   | HEXTORAW decode | unit | BLOB / XML bytes | 
`TransactionCommitConsumerTest` | Pass |
   | Re-select SQL + flashback fallback | unit | CLOB / XMLTYPE | 
`OracleLobReselectSqlTest` | Pass |
   | Placeholder `warn_and_keep` / `null` / `fail` | unit | LOB handler | 
`OracleLobUnavailableValueHandlerTest` | Pass |
   | JDBC CLOB / BLOB / XMLTYPE converters | unit | snapshot JDBC | 
`OracleJdbcLobValueConvertersTest`, `OracleXmlTypeJdbcTest` | Pass |
   | Option validation (`lob.enabled` vs reselect / handling) | unit | config | 
`OracleIncrementalSourceFactoryTest` | Pass |
   | Reader close of AutoCloseable deserialize schema | unit | CDC base | 
`IncrementalSourceReaderTest` (19 tests) | Pass |
   | Snapshot CLOB / NCLOB / BLOB / XMLTYPE (emoji, PNG, MP4) | snapshot | JDBC 
converters | XE 11g (`gvenzl/oracle-xe:11-full`), `-Doracle.xe.it=true` | Pass |
   | Snapshot CLOB / NCLOB / BLOB / XMLTYPE (emoji, PNG, MP4) | snapshot | JDBC 
converters | XE 21c (`gvenzl/oracle-xe:21-slim-faststart`), 
`-Doracle.xe.it=true` | Pass |
   
   Default `mvn test` skips the two XE rows so CI does not pull Oracle XE.
   
   ### Check list
   
   * [x] New jars: `xdb` / `xmlparserv2` are `provided` and are not bundled
   * [x] Docs updated (`docs/en`, `docs/zh`)
   * [x] `incompatible-changes.md` (defaults unchanged)
   * [x] Existing connector: plugin-mapping / dist / labeler / plugin_config 
not required


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to