[
https://issues.apache.org/jira/browse/GOBBLIN-1116?focusedWorklogId=424408&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-424408
]
ASF GitHub Bot logged work on GOBBLIN-1116:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 17/Apr/20 20:40
Start Date: 17/Apr/20 20:40
Worklog Time Spent: 10m
Work Description: sv2000 commented on pull request #2956: GOBBLIN-1116:
Avoid registering schema with schema registry during Me…
URL: https://github.com/apache/incubator-gobblin/pull/2956#discussion_r410459080
##########
File path:
gobblin-modules/gobblin-kafka-common/src/main/java/org/apache/gobblin/metrics/reporter/util/SchemaRegistryVersionWriter.java
##########
@@ -41,58 +44,50 @@
* {@link org.apache.gobblin.metrics.kafka.KafkaAvroSchemaRegistry} to get
Schema version identifier and write it to
* {@link java.io.DataOutputStream}.
*/
+@Slf4j
public class SchemaRegistryVersionWriter implements
SchemaVersionWriter<Schema> {
private final KafkaAvroSchemaRegistry registry;
private Map<Schema, String> registrySchemaIds;
- private final Optional<String> overrideName;
- private final Optional<Schema> schema;
- private final Optional<String> schemaId;
+ private final String overrideName;
+ private final Schema schema;
+ private final String schemaId;
private final int schemaIdLengthBytes;
public SchemaRegistryVersionWriter(Config config)
- throws IOException {
- this(new KafkaAvroSchemaRegistry(ConfigUtils.configToProperties(config)),
Optional.<String>absent(),
- Optional.<Schema>absent());
- }
-
- public SchemaRegistryVersionWriter(KafkaAvroSchemaRegistry registry, String
overrideName)
- throws IOException {
- this(registry, overrideName, Optional.<Schema>absent());
+ throws SchemaRegistryException {
+ this(new KafkaAvroSchemaRegistry(ConfigUtils.configToProperties(config)),
null, null, null);
}
- public SchemaRegistryVersionWriter(KafkaAvroSchemaRegistry registry, String
overrideName,
- Optional<Schema> singleSchema)
- throws IOException {
- this(registry, Optional.of(overrideName), singleSchema);
+ public SchemaRegistryVersionWriter(KafkaAvroSchemaRegistry registry,
@Nullable String overrideName, @Nullable Schema singleSchema)
Review comment:
Added constructor to preserve backwards compatibility.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 424408)
Time Spent: 2h 50m (was: 2h 40m)
> Avoid registering schema with schema registry during MetricReporting
> initialization from cluster workers
> --------------------------------------------------------------------------------------------------------
>
> Key: GOBBLIN-1116
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1116
> Project: Apache Gobblin
> Issue Type: Improvement
> Components: gobblin-cluster
> Affects Versions: 0.15.0
> Reporter: Sudarshan Vasudevan
> Assignee: Hung Tran
> Priority: Major
> Fix For: 0.15.0
>
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> This PR avoids having to register schema with schema registry on Kafka metric
> reporter initialization from cluster workers. This avoids a potentially large
> number of connections that may be initiated with Kafka Schema registry on
> Gobblin on Yarn application launch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
