[
https://issues.apache.org/jira/browse/SAMZA-82?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13818056#comment-13818056
]
Jakob Homan commented on SAMZA-82:
----------------------------------
bq. Recommended fix: I think what we need to do is make TaskInstance only
register valid partitions
Yes, this is what I'm saying. I'm saying that trying to register partitions
based on the maximum is wrong.
> Not use maximum number of partitions when initializing streams
> --------------------------------------------------------------
>
> Key: SAMZA-82
> URL: https://issues.apache.org/jira/browse/SAMZA-82
> Project: Samza
> Issue Type: Bug
> Components: kafka
> Affects Versions: 0.7.0
> Reporter: Jakob Homan
> Assignee: Jakob Homan
> Fix For: 0.7.0
>
>
> Util.scala:
> {code} /**
> * Uses config to create SystemAdmin classes for all input stream systems to
> * get each input stream's partition count, then returns the maximum count.
> * An input stream with two partitions, and a second input stream with four
> * partitions would result in this method returning 4.
> */
> def getMaxInputStreamPartitions(config: Config) = {
> {code}
> This approach works if all the streams have the same number of partitions,
> but is inefficient for other cases and, where the underlying system gets
> cranky about being asked about non-existing partitions, fails. We should
> eagerly figure out the correct number of partitions for each topic and pass
> that information from there.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
