odbozhou commented on code in PR #293:
URL: https://github.com/apache/rocketmq-connect/pull/293#discussion_r972803807
##########
connectors/rocketmq-connect-jms/src/main/java/org/apache/rocketmq/connect/jms/connector/BaseJmsSourceTask.java:
##########
@@ -135,12 +123,52 @@ public ByteBuffer getMessageContent(Message message)
throws JMSException {
while ((i = streamMessage.readBytes(by)) != -1) {
bis.write(by, 0, i);
}
- data = bis.toByteArray();
+ data = bis.toString();
} else {
- // The exception is printed and does not need to be written as a
DataConnectException
throw new RuntimeException("message type exception");
}
- return ByteBuffer.wrap(data);
+ return data;
+ }
+
+ private ConnectRecord message2ConnectRecord(Message message) throws
JMSException {
+ Schema schema = SchemaBuilder.struct().name("jms").build();
+ final List<Field> fields = buildFields();
+ schema.setFields(fields);
+ return new ConnectRecord(buildRecordPartition(),
+ buildRecordOffset(),
+ System.currentTimeMillis(),
+ schema,
+ buildPayLoad(fields, message, schema));
+ }
+
+ private RecordOffset buildRecordOffset() {
+ Map<String, Long> offsetMap = new HashMap<>();
Review Comment:
jm and activemq should also have site-related information, so that the
progress of consumption can be recorded. RecordOffset is mainly used to save
site information, so as to realize the function of resuming transmission from
breakpoints.
##########
connectors/rocketmq-connect-jms/src/main/java/org/apache/rocketmq/connect/jms/connector/BaseJmsSourceTask.java:
##########
@@ -135,12 +123,52 @@ public ByteBuffer getMessageContent(Message message)
throws JMSException {
while ((i = streamMessage.readBytes(by)) != -1) {
bis.write(by, 0, i);
}
- data = bis.toByteArray();
+ data = bis.toString();
} else {
- // The exception is printed and does not need to be written as a
DataConnectException
throw new RuntimeException("message type exception");
}
- return ByteBuffer.wrap(data);
+ return data;
+ }
+
+ private ConnectRecord message2ConnectRecord(Message message) throws
JMSException {
+ Schema schema = SchemaBuilder.struct().name("jms").build();
+ final List<Field> fields = buildFields();
+ schema.setFields(fields);
+ return new ConnectRecord(buildRecordPartition(),
+ buildRecordOffset(),
+ System.currentTimeMillis(),
+ schema,
+ buildPayLoad(fields, message, schema));
+ }
+
+ private RecordOffset buildRecordOffset() {
+ Map<String, Long> offsetMap = new HashMap<>();
+ RecordOffset recordOffset = new RecordOffset(offsetMap);
+ return recordOffset;
+ }
+
+ private RecordPartition buildRecordPartition() {
+ Map<String, String> partitionMap = new HashMap<>();
+ partitionMap.put("partition", config.getBrokerUrl());
+ RecordPartition recordPartition = new RecordPartition(partitionMap);
+ return recordPartition;
+ }
+
+ private List<Field> buildFields() {
+ final Schema stringSchema = SchemaBuilder.string().build();
+ List<Field> fields = new ArrayList<>();
+ fields.add(new Field(0, Config.DESTINATION_NAME, stringSchema));
+ fields.add(new Field(1, Config.DESTINATION_TYPE, stringSchema));
+ fields.add(new Field(2, Config.MESSAGE, stringSchema));
+ return fields;
+ }
+
+ private Struct buildPayLoad(List<Field> fields, Message message, Schema
schema) throws JMSException {
+ Struct payLoad = new Struct(schema);
+ payLoad.put(fields.get(0), config.getDestinationName());
+ payLoad.put(fields.get(1), config.getDestinationType());
+ payLoad.put(fields.get(2), getMessageContent(message));
Review Comment:
It is recommended that the payload only saves the data in the message, and
the valueschema is used to describe the format of the data
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
