2019-07-05 09:17:57 UTC - Guillaume Rosauro: Hi Team. I am trying Pulsar with Kafka-connect/Debezium. It works as expected except for Kafka-connect SMT (<http://transforms.xxx|transforms.xxx>). Unfortunately without SMT, Debezium connector and messages are quite unusable (we need to unwrap the Enveloppe). According to @tuteng "those parameters are not taken into account (eventually by Debezium)" I would like to contribute to add those SMT parameters to kafka connect but I do not know where to start and the best way to achieve this. Any help would be appreciate. ---- 2019-07-05 09:18:08 UTC - tuteng: @tuteng has joined the channel ---- 2019-07-05 09:20:05 UTC - Ali Ahmed: @Guillaume Rosauro which specific transformation do you need ? ---- 2019-07-05 09:21:16 UTC - Guillaume Rosauro: @Ali Ahmed: UnwrapEnvelope for example (<https://github.com/debezium/debezium/blob/master/debezium-core/src/main/java/io/debezium/transforms/UnwrapFromEnvelope.java>) ---- 2019-07-05 09:22:14 UTC - Guillaume Rosauro: or RegexRouter : <https://docs.confluent.io/current/connect/transforms/regexrouter.html> ---- 2019-07-05 10:12:38 UTC - tuteng: ExtractNewRecordState is used to replace the UnwrapFromEnvelope in the new version. <https://github.com/debezium/debezium/blob/master/debezium-core/src/main/java/io/debezium/transforms/ExtractNewRecordState.java>. As mentioned in the note, after this conversion, you need to give Pulsar a signal to make Pulsar do the compaction operation. The above is my understanding @Guillaume Rosauro ---- 2019-07-05 10:13:42 UTC - Guillaume Rosauro: @tuteng you are right, but it does not really matter. No one works. ---- 2019-07-05 10:59:20 UTC - pradeep: Hi Team, Does pulsar support any stickness of topic to a single DC cluster (even replication is enabled) ? I mean for a given topic all read and write should happen to that given DC cluster where ever the topic got created first? ---- 2019-07-05 12:19:23 UTC - Guillaume Rosauro: @tuteng I am sorry but I do not really understand the link between a Kafka Connect Tranformation and the Pulsar Compaction Operation. Could you explain me more about that ? ---- 2019-07-05 12:39:28 UTC - tuteng: I am not a kafka expert. I read some notes. the class ExtractNewRecordState mainly processes two parameters, tombstones and addOperationHeader, which are related to the compacttion of kafka, <http://cloudurable.com/blog/kafka-architecture-log-compaction/index.html>, Therefore, if this class ExtractNewRecordState is to be processed in Pulsar, the correct method is to trigger Pulsar's compaction after receiving this parameter. Of course, this is for the conversion of ExtractNewRecordState, and others transforms may have different processing logic. ---- 2019-07-05 16:08:12 UTC - Matteo Merli: @pradeep with async replication, you chose where to write by using a different “serviceURL” ---- 2019-07-05 22:42:50 UTC - t: @t has joined the channel ----
