ilyasahsan123 commented on PR #40785:
URL: https://github.com/apache/beam/pull/40785#issuecomment-6419987505
Thanks for catching this. You're right, the transaction block should not
depend on `fullMetadata`. I've moved it so the output now matches the Debezium
payload. The block is only filled in when the user sets
`provide.transaction.metadata=true`, so the default output does not change.
To support this, I added two new types:
- `Transaction`: added to each change event, so the event can be linked to
its transaction.
- `TransactionEvent`: the BEGIN and END events, emitted only when
`provide.transaction.metadata=true`.
**With `provide.transaction.metadata=true`**
BEGIN event:
```json
{
"status" : "BEGIN",
"id" : "781:39950456",
"ts_ms" : 1793024470210,
"event_count" : null,
"data_collections" : null
}
```
END event:
```json
{
"status" : "END",
"id" : "781:39951784",
"ts_ms" : 1793024470210,
"event_count" : 3,
"data_collections" : [ {
"data_collection" : "inventory.customers",
"event_count" : 3
} ]
}
```
Change event (note the `transaction` block at the end):
```json
{
"metadata" : {
"connector" : "postgresql",
"version" : "3.5.2.Final",
"name" : "beam-debezium-connector",
"database" : "inventory",
"schema" : "inventory",
"table" : "customers",
"op" : "c",
"ts_ms" : 1793024873511,
"snapshot" : "false",
"sequence" : "[null,\"39950456\"]",
"ts_us" : 1793024873511455,
"ts_ns" : 1793024873511455000,
"txId" : 781,
"lsn" : 39950456,
"xmin" : null,
"origin" : null,
"origin_lsn" : null
},
"before" : null,
"after" : {
"fields" : {
"last_name" : "One",
"id" : 5001,
"first_name" : "Ann",
"email" : "[email protected]"
}
},
"transaction" : {
"id" : "781:39950456",
"total_order" : 1,
"data_collection_order" : 1
}
}
```
**With `provide.transaction.metadata=false` (default)**
No BEGIN/END events are emitted, and `transaction` is `null`:
```json
{
"metadata" : {
"connector" : "postgresql",
"version" : "3.5.2.Final",
"name" : "beam-debezium-connector",
"database" : "inventory",
"schema" : "inventory",
"table" : "customers",
"op" : "c",
"ts_ms" : 1793025206974,
"snapshot" : "false",
"sequence" : "[null,\"39950456\"]",
"ts_us" : 1793025206974908,
"ts_ns" : 1793025206974908000,
"txId" : 781,
"lsn" : 39950456,
"xmin" : null,
"origin" : null,
"origin_lsn" : null
},
"before" : null,
"after" : {
"fields" : {
"last_name" : "One",
"id" : 5001,
"first_name" : "Ann",
"email" : "[email protected]"
}
},
"transaction" : null
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
