Vladislav Sergeev commented on SAMZA-1035:

Hello,Jake Maes .

As i understand logic by default KafkaSystemProducer creates an instance of 
Producer with acks=1.

For generalization i suppose that SystemProducer should be parameterized and 
archetecture should be a little change(i mean some interfaces: 
I suppose that SystemProducer interface should have all methods parameterized, 
because on creation of producers for some systems it would be usefull to method 
"void start()" return something:) 

As you said if we will track flush we need to change the interface(i don't 
consider the custom realization of interfaces)

P.S. sorry for so late answer.

> Return offset of produced message
> ---------------------------------
>                 Key: SAMZA-1035
>                 URL: https://issues.apache.org/jira/browse/SAMZA-1035
>             Project: Samza
>          Issue Type: Wish
>            Reporter: Vladislav Sergeev
>            Priority: Minor
> Hello.
> We use Apache Kafka as a databus and Apache Samza as a router.
> I suppose good feature for samza will be returning offset of a message ,in 
> the case when we transfer messages from topic A to topic B.
> In our case we have SystemA that produce messages to input_topic and we route 
> with some transformations that messages to topicA and topicB.
> Sometimes it is usefull to collect metrics of processed messages.
> We can do it when they come to method :
> process(IncomingMessageEnvelope incomingMessageEnvelope, MessageCollector 
> messageCollector, TaskCoordinator taskCoordinator)
> But when we route them to another topic like this:
> messageCollector.send(new OutgoingMessageEnvelope(new SystemStream("kafka", 
> topic), keyOut, message));
> We don't know what offset has message in routed topic.
> As i saw in your code:
> https://github.com/apache/samza/blob/master/samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemProducer.scala
> You use kafka producer that returns offsets for sent message.So it will be 
> great to get it.

This message was sent by Atlassian JIRA

Reply via email to