On Tue, Jul 30, 2019 at 10:33 AM Omar Al-Safi <[email protected]> wrote: > > Thanks Claus for your reply. What I meant about the raw format, is the > format that Debezium uses to store event record values which of type > KafkaConnect > Struct > <https://kafka.apache.org/11/javadoc/org/apache/kafka/connect/data/Struct.html>. > DBZ contains the CDC event data into KafkaConnect SourceRecord > <https://kafka.apache.org/11/javadoc/org/apache/kafka/connect/source/SourceRecord.html> > parent which holds > information such as the key, value, schema, DDL statement, topic, offsets > ..etc. Apart from the value, I am storing the other information in the > message header (record key, topic name, source offset and timestamp) and of > course storing the value in the message body, which is the record value is > of type KafkaConnect Struct > <https://kafka.apache.org/11/javadoc/org/apache/kafka/connect/data/Struct.html> > (as > well for the key) as I mentioned earlier. > > So then from my understanding, then is the responsibility of the user to > transform the data omitted from DBZ camel component into the shape he/she > desires to process further downstream. >
Yes and the user would need to do that as well if he/she was using Debezium without the Camel component. > On Tue, 30 Jul 2019 at 05:17, Claus Ibsen <[email protected]> wrote: > > > Hi > > > > No there is no strict rule or something, each component can do what is > > most appropriate for it. > > Sometimes a component has options users can configure on the endpoint > > etc that can dictate the format, for example binary, json, pojo, xml > > etc. > > > > When you say raw format from debezium what java type does it use, and > > is the data in some kind of structure or binary format or what? > > > > > > On Mon, Jul 29, 2019 at 11:50 AM Omar Al-Safi <[email protected]> wrote: > > > > > > Hello Camel devs, > > > > > > I am relatively new to Apache Camel. I am developing a new component to > > > Apache Camel that adds the support of https://debezium.io/ to Camel. > > > My question is about the message data format when I save the message body > > > value into the Message (message.setBody(value)), I was confused if there > > is > > > a strict message format that I should parse my value before saving into > > the > > > exchange message body or I just can save the raw value directly without > > > parsing it, is this correct? I was looking at some examples such Kinesis > > > < > > https://github.com/apache/camel/blob/master/components/camel-aws-kinesis/src/main/java/org/apache/camel/component/aws/kinesis/KinesisEndpoint.java#L94 > > > > > > and > > > Kafka > > > < > > https://github.com/apache/camel/blob/master/components/camel-kafka/src/main/java/org/apache/camel/component/kafka/KafkaEndpoint.java#L172 > > > > > > and > > > as I understood here, the raw type value was saved directly into the > > > exchange body, however I wanted hear if there are any recommendation on > > the > > > format when the data being saved into the exchange body message. > > > > > > Thank you, > > > Omar > > > > > > > > -- > > Claus Ibsen > > ----------------- > > http://davsclaus.com @davsclaus > > Camel in Action 2: https://www.manning.com/ibsen2 > > -- Claus Ibsen ----------------- http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
