Cool, thanks for the information On Tue, 30 Jul 2019 at 11:53, Claus Ibsen <[email protected]> wrote:
> On Tue, Jul 30, 2019 at 10:33 AM Omar Al-Safi <[email protected]> wrote: > > > > Thanks Claus for your reply. What I meant about the raw format, is the > > format that Debezium uses to store event record values which of type > > KafkaConnect > > Struct > > < > https://kafka.apache.org/11/javadoc/org/apache/kafka/connect/data/Struct.html > >. > > DBZ contains the CDC event data into KafkaConnect SourceRecord > > < > https://kafka.apache.org/11/javadoc/org/apache/kafka/connect/source/SourceRecord.html > > > > parent which holds > > information such as the key, value, schema, DDL statement, topic, offsets > > ..etc. Apart from the value, I am storing the other information in the > > message header (record key, topic name, source offset and timestamp) and > of > > course storing the value in the message body, which is the record value > is > > of type KafkaConnect Struct > > < > https://kafka.apache.org/11/javadoc/org/apache/kafka/connect/data/Struct.html > > > > (as > > well for the key) as I mentioned earlier. > > > > So then from my understanding, then is the responsibility of the user to > > transform the data omitted from DBZ camel component into the shape he/she > > desires to process further downstream. > > > > Yes and the user would need to do that as well if he/she was using > Debezium without the Camel component. > > > > On Tue, 30 Jul 2019 at 05:17, Claus Ibsen <[email protected]> wrote: > > > > > Hi > > > > > > No there is no strict rule or something, each component can do what is > > > most appropriate for it. > > > Sometimes a component has options users can configure on the endpoint > > > etc that can dictate the format, for example binary, json, pojo, xml > > > etc. > > > > > > When you say raw format from debezium what java type does it use, and > > > is the data in some kind of structure or binary format or what? > > > > > > > > > On Mon, Jul 29, 2019 at 11:50 AM Omar Al-Safi <[email protected]> > wrote: > > > > > > > > Hello Camel devs, > > > > > > > > I am relatively new to Apache Camel. I am developing a new component > to > > > > Apache Camel that adds the support of https://debezium.io/ to Camel. > > > > My question is about the message data format when I save the message > body > > > > value into the Message (message.setBody(value)), I was confused if > there > > > is > > > > a strict message format that I should parse my value before saving > into > > > the > > > > exchange message body or I just can save the raw value directly > without > > > > parsing it, is this correct? I was looking at some examples such > Kinesis > > > > < > > > > https://github.com/apache/camel/blob/master/components/camel-aws-kinesis/src/main/java/org/apache/camel/component/aws/kinesis/KinesisEndpoint.java#L94 > > > > > > > > and > > > > Kafka > > > > < > > > > https://github.com/apache/camel/blob/master/components/camel-kafka/src/main/java/org/apache/camel/component/kafka/KafkaEndpoint.java#L172 > > > > > > > > and > > > > as I understood here, the raw type value was saved directly into the > > > > exchange body, however I wanted hear if there are any recommendation > on > > > the > > > > format when the data being saved into the exchange body message. > > > > > > > > Thank you, > > > > Omar > > > > > > > > > > > > -- > > > Claus Ibsen > > > ----------------- > > > http://davsclaus.com @davsclaus > > > Camel in Action 2: https://www.manning.com/ibsen2 > > > > > > > -- > Claus Ibsen > ----------------- > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2 >
