voonhous commented on code in PR #19760:
URL: https://github.com/apache/hudi/pull/19760#discussion_r3870155532
##########
hudi-utilities/src/test/java/org/apache/hudi/utilities/sources/TestAvroKafkaSource.java:
##########
@@ -151,6 +159,31 @@ void sendMessagesToKafkaWithNullKafkaValue(String topic,
int count, int numParti
}
}
+ private void sendUserRecordsWithConfluentSerializer(String topic, String
registryUrl, HoodieSchema schema, int count, String phone) {
+ Properties config = getProducerProperties();
+ config.put("value.serializer", KafkaAvroSerializer.class.getName());
+ config.put("schema.registry.url", registryUrl);
+ config.put("auto.register.schemas", "true");
+ try (Producer<String, GenericRecord> producer = new
KafkaProducer<>(config)) {
+ for (int i = 0; i < count; i++) {
+ GenericRecordBuilder builder = new
GenericRecordBuilder(schema.toAvroSchema())
+ .set("name", "user" + i)
+ .set("favorite_number", i)
+ .set("favorite_color", "blue");
+ if (phone != null) {
+ builder.set("phone", phone);
+ }
+ producer.send(new ProducerRecord<>(topic, "key", builder.build()));
+ }
+ }
+ }
+
+ private HoodieSchema loadSchemaFromResource(String resourcePath) throws
IOException {
Review Comment:
Switched to `SchemaTestUtil.getSchemaFromResource` in 95aa4002f492; the
private loader and the `throws IOException` are gone.
##########
hudi-utilities/src/test/java/org/apache/hudi/utilities/sources/TestAvroKafkaSource.java:
##########
@@ -311,6 +344,59 @@ void testConfigureSchemaDeserializer() throws IOException {
assertEquals(StringUtils.concatenateWithThreshold(String.format("%s_",
groupId), schemaHash, GROUP_ID_MAX_BYTES_LENGTH), newGroupId);
}
+ /**
+ * Covers the per-fetch re-configure branch of {@link
AvroKafkaSource#readFromCheckpoint}, which regressed twice
Review Comment:
Reworded in 95aa4002f492: the javadoc now names #10118 (re-stamp the
provider's current schema before every read) and #12111 (`group.id` rotation)
as the two mechanisms, and the PR description says the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
