tkaymak commented on issue #34747: URL: https://github.com/apache/beam/issues/34747#issuecomment-2991516502
Thank you @Abacn! I have added some more logging and found that the additional debezium connection properties get lost when being passed from Python to the Java Expansion Service. (`Raw Connection Properties` are empty.) My plan is now to allow passing a JSON string for these properties and overloading the Java class to be backwards compatible. ``` INFO: Expanding 'Read from debezium/ExternalTransform(beam:transform:org.apache.beam:debezium_read:v1)' with URN 'beam:transform:org.apache.beam:debezium_read:v1' Jun 20, 2025 12:23:16 PM org.apache.beam.sdk.expansion.service.ExpansionService payloadToConfig WARNING: Configuration class 'org.apache.beam.io.debezium.DebeziumTransformRegistrar$ReadBuilder$Configuration' has no schema registered. Attempting to construct with setter approach. Jun 20, 2025 12:23:17 PM org.apache.beam.sdk.expansion.service.ExpansionService payloadToConfig WARNING: Configuration class 'org.apache.beam.io.debezium.DebeziumTransformRegistrar$ReadBuilder$Configuration' has no schema registered. Attempting to construct with setter approach. Jun 20, 2025 12:23:17 PM org.apache.beam.io.debezium.DebeziumIO$ConnectorConfiguration getConfigurationMap INFO: --- RAW DEBEZIUM CONFIGURATION --- Jun 20, 2025 12:23:17 PM org.apache.beam.io.debezium.DebeziumIO$ConnectorConfiguration getConfigurationMap INFO: Raw Connector Class: class io.debezium.connector.postgresql.PostgresConnector Jun 20, 2025 12:23:17 PM org.apache.beam.io.debezium.DebeziumIO$ConnectorConfiguration getConfigurationMap INFO: Raw HostName: localhost Jun 20, 2025 12:23:17 PM org.apache.beam.io.debezium.DebeziumIO$ConnectorConfiguration getConfigurationMap INFO: Raw Port: 49211 Jun 20, 2025 12:23:17 PM org.apache.beam.io.debezium.DebeziumIO$ConnectorConfiguration getConfigurationMap INFO: Raw Username: debezium Jun 20, 2025 12:23:17 PM org.apache.beam.io.debezium.DebeziumIO$ConnectorConfiguration getConfigurationMap INFO: Raw Connection Properties: Jun 20, 2025 12:23:17 PM org.apache.beam.io.debezium.DebeziumIO$ConnectorConfiguration getConfigurationMap INFO: --- FINAL DEBEZIUM CONFIGURATION --- connector.class -> io.debezium.connector.postgresql.PostgresConnector topic.prefix -> beam-debezium-connector database.user -> debezium schema.history.internal.kafka.topic -> schema-changes.inventory database.hostname -> localhost database.password -> dbz schema.history.internal.kafka.bootstrap.servers -> localhost:9092 database.history -> org.apache.beam.io.debezium.KafkaSourceConsumerFn$DebeziumSDFDatabaseHistory database.port -> 49211 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
