[
https://issues.apache.org/jira/browse/SAMZA-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14018060#comment-14018060
]
Chris Riccomini commented on SAMZA-144:
---------------------------------------
Hmm. You raise a good point about the broker list/zk overrides. I don't think
we can support these overrides since it kind of violates what it means to be a
"system" in Samza.
I think the use case for this is more around tweaking things like compression,
batch size, etc. What do you think about just dis-allowing illegal overrides
from being sent (e.g. broker.metadata.list, etc) by throwing an exception?
> Add stream-level overrides for KafkaSystemFactory config
> --------------------------------------------------------
>
> Key: SAMZA-144
> URL: https://issues.apache.org/jira/browse/SAMZA-144
> Project: Samza
> Issue Type: Bug
> Components: kafka
> Affects Versions: 0.6.0
> Reporter: Chris Riccomini
> Assignee: Yan Fang
>
> Currently, we allow Kafka systems to be defined with:
> {noformat}
> systems.my-kafka-system.consumer.*
> {noformat}
> And:
> {noformat}
> systems.my-kafka-system.producer.*
> {noformat}
> We don't support stream-level configurations, the way we do with StreamConfig
> configuration:
> {code}
> val STREAM_PREFIX = "systems.%s.streams.%s."
> val MSG_SERDE = STREAM_PREFIX + "samza.msg.serde"
> val KEY_SERDE = STREAM_PREFIX + "samza.key.serde"
> val CONSUMER_RESET_OFFSET = STREAM_PREFIX + "samza.reset.offset"
> {code}
> We should add stream-level configuration override support for Kafka consumer
> and producers with:
> {noformat}
> systems.my-kafka-system.streams.my-kafka-topic.consumer.*
> systems.my-kafka-system.streams.my-kafka-topic.producer.*
> {noformat}
> KafkaConfig and KafkaSystemFactory should be updated to handle these.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
