[ https://issues.apache.org/jira/browse/SAMZA-952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Branislav Cogic updated SAMZA-952: ---------------------------------- Attachment: SAMZA-952_0.patch > Add validation for broadcast inputs configured for partitions that don't exist > ------------------------------------------------------------------------------ > > Key: SAMZA-952 > URL: https://issues.apache.org/jira/browse/SAMZA-952 > Project: Samza > Issue Type: Bug > Affects Versions: 0.10.0 > Reporter: Jake Maes > Assignee: Branislav Cogic > Attachments: SAMZA-952_0.patch > > > We had an issue at LinkedIn where the user configured a broadcast input as: > {noformat} > task.broadcast.inputs=kafka-queueing.ads_glob#[0-9] > {noformat} > But the ads_glob topic only has 8 partitions. > This caused the following exception in the container log: > {noformat} > 2016-05-24 19:59:20 SamzaContainer [ERROR] Caught exception in process loop. > org.apache.samza.SamzaException: No metadata available for partition null. > at > org.apache.samza.checkpoint.OffsetManager$$anonfun$loadDefaults$1$$anonfun$apply$15.apply(OffsetManager.scala:412) > at > org.apache.samza.checkpoint.OffsetManager$$anonfun$loadDefaults$1$$anonfun$apply$15.apply(OffsetManager.scala:380) > at scala.collection.mutable.HashSet.foreach(HashSet.scala:79) > at > org.apache.samza.checkpoint.OffsetManager$$anonfun$loadDefaults$1.apply(OffsetManager.scala:380) > at > org.apache.samza.checkpoint.OffsetManager$$anonfun$loadDefaults$1.apply(OffsetManager.scala:378) > at > scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98) > at > scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98) > at > scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226) > at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39) > at scala.collection.mutable.HashMap.foreach(HashMap.scala:98) > at > org.apache.samza.checkpoint.OffsetManager.loadDefaults(OffsetManager.scala:378) > at > org.apache.samza.checkpoint.OffsetManager.start(OffsetManager.scala:175) > at > org.apache.samza.container.SamzaContainer.startOffsetManager(SamzaContainer.scala:604) > at > org.apache.samza.container.SamzaContainer.run(SamzaContainer.scala:550) > at > org.apache.samza.container.SamzaContainer$.safeMain(SamzaContainer.scala:93) > at > org.apache.samza.container.SamzaContainer$.main(SamzaContainer.scala:67) > at org.apache.samza.container.SamzaContainer.main(SamzaContainer.scala) > {noformat} > We should add some validation and a meaningful error message for cases when > the broadcast inputs don't make sense compared to the metadata. -- This message was sent by Atlassian JIRA (v6.3.4#6332)