tkaymak commented on code in PR #41331: URL: https://github.com/apache/beam/pull/41331#discussion_r4601008809
########## CHANGES.md: ########## @@ -89,6 +89,7 @@ * Support for X source added (Java/Python) ([#X](https://github.com/apache/beam/issues/X)). * Upgraded Iceberg dependency to 1.12.0 ([#40971](https://github.com/apache/beam/issues/40971)). * (Java/Python/Go) DebeziumIO: `readAsJson()` now emits Debezium transaction metadata with `provide.transaction.metadata=true` (Python/Go via `connection_properties`) ([#40784](https://github.com/apache/beam/issues/40784)). +* (Java/Python/Go) DebeziumIO: `readAsJson()` now emits MySQL, Oracle and SQL Server schema change events with `include.schema.changes=true` (Python/Go via `connection_properties`) ([#41330](https://github.com/apache/beam/issues/41330)). Review Comment: The wording with `include.schema.changes=true` reads as opt in, but true is the Debezium default. With the flag from the summary, this line can name it, for example `withSchemaChangeEvents(true)` together with `include.schema.changes=true`. The same wording is in `README.md:100`, `debezium.py:155` and `debezium.go:151`, where the Python and Go docs would name `schema_change_events` and `SchemaChangeEvents()`. ########## sdks/java/io/debezium/src/main/java/org/apache/beam/io/debezium/SourceRecordJson.java: ########## @@ -152,10 +167,17 @@ public SourceRecordJson(@Nullable SourceRecord sourceRecord, boolean fullMetadat if (this.value == null) { this.event = new Event(null, null, null, null); this.transactionEvent = null; + this.changeSchemaEvent = null; this.metadataExtras = Collections.emptyMap(); } else if (isTransactionMetadataEvent(sourceRecord.keySchema())) { this.event = null; this.transactionEvent = this.loadTransactionEvent(); + this.changeSchemaEvent = null; + this.metadataExtras = Collections.emptyMap(); + } else if (isSchemaChangeEventKeyName(sourceRecord.keySchema())) { Review Comment: On master these records fall through to the data branch and are emitted with `metadata` and null `before` and `after`. They are not skipped. Behind the flag, please keep `metadata`, `before` and `after` in this payload, so `withFullMetadata(true)` still applies and `snapshot`, `file` and `pos` are not lost. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
