[
https://issues.apache.org/jira/browse/GOBBLIN-758?focusedWorklogId=237129&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-237129
]
ASF GitHub Bot logged work on GOBBLIN-758:
------------------------------------------
Author: ASF GitHub Bot
Created on: 04/May/19 00:29
Start Date: 04/May/19 00:29
Worklog Time Spent: 10m
Work Description: zxcware commented on pull request #2622: [GOBBLIN-758]
Added new reporters to emit MetricReport and GobblinTrackingEvent without
serializing them. Also added random key generator for reporters.
URL: https://github.com/apache/incubator-gobblin/pull/2622#discussion_r280961171
##########
File path:
gobblin-modules/gobblin-kafka-common/src/main/java/org/apache/gobblin/metrics/kafka/PusherUtils.java
##########
@@ -39,8 +41,26 @@ public static Pusher getPusher(String pusherClassName,
String brokers, String to
try {
Class<?> pusherClass = Class.forName(pusherClassName);
- return (Pusher)
GobblinConstructorUtils.invokeLongestConstructor(pusherClass,
- brokers, topic, config);
+ return (Pusher)
GobblinConstructorUtils.invokeLongestConstructor(pusherClass, brokers, topic,
config);
+ } catch (ReflectiveOperationException e) {
+ throw new RuntimeException("Could not instantiate kafka pusher", e);
+ }
+ }
+
+ /**
+ * Create a {@link KeyValuePusher}
+ * @param pusherClassName the {@link Pusher} class to instantiate
+ * @param brokers brokers to connect to
+ * @param topic the topic to write to
+ * @param config additional configuration for configuring the {@link Pusher}
+ * @return a {@link KeyValuePusher}
+ */
+ public static KeyValuePusher getKeyValuePusher(String pusherClassName,
String brokers, String topic,
Review comment:
This is redundant and can be removed. A `KeyValuePusher` is a `Pusher`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 237129)
Time Spent: 1h 20m (was: 1h 10m)
> Use keys to push messages to kafka. Implemented Metric and Event reporters
> that push messages without serialization.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: GOBBLIN-758
> URL: https://issues.apache.org/jira/browse/GOBBLIN-758
> Project: Apache Gobblin
> Issue Type: Improvement
> Reporter: Vikram Bohra
> Priority: Major
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> This ticket introduces the following changes.
> 1) It reduces the number of connections to Kafka by using a randomly
> generated key for each instance of the reporter.
> 2) Introduces a KeyValuePusher interface that provides a cleaner way to push
> key value messages to kafka.
> 3) Default implementation of the above interface for kafka.
> 4) New Reporters for reporting GobblinTrackingEvent and MetricReport without
> serializing them.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
