[ 
https://issues.apache.org/jira/browse/SAMZA-952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Branislav Cogic reassigned SAMZA-952:
-------------------------------------

    Assignee: Branislav Cogic

> Add validation for broadcast inputs configured for partitions that don't exist
> ------------------------------------------------------------------------------
>
>                 Key: SAMZA-952
>                 URL: https://issues.apache.org/jira/browse/SAMZA-952
>             Project: Samza
>          Issue Type: Bug
>    Affects Versions: 0.10.0
>            Reporter: Jake Maes
>            Assignee: Branislav Cogic
>
> We had an issue at LinkedIn where the user configured a broadcast input as: 
> {noformat}
> task.broadcast.inputs=kafka-queueing.ads_glob#[0-9]
> {noformat}
> But the ads_glob topic only has 8 partitions. 
> This caused the following exception in the container log:
> {noformat}
> 2016-05-24 19:59:20 SamzaContainer [ERROR] Caught exception in process loop.
> org.apache.samza.SamzaException: No metadata available for partition null.
>       at 
> org.apache.samza.checkpoint.OffsetManager$$anonfun$loadDefaults$1$$anonfun$apply$15.apply(OffsetManager.scala:412)
>       at 
> org.apache.samza.checkpoint.OffsetManager$$anonfun$loadDefaults$1$$anonfun$apply$15.apply(OffsetManager.scala:380)
>       at scala.collection.mutable.HashSet.foreach(HashSet.scala:79)
>       at 
> org.apache.samza.checkpoint.OffsetManager$$anonfun$loadDefaults$1.apply(OffsetManager.scala:380)
>       at 
> org.apache.samza.checkpoint.OffsetManager$$anonfun$loadDefaults$1.apply(OffsetManager.scala:378)
>       at 
> scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
>       at 
> scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
>       at 
> scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226)
>       at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
>       at scala.collection.mutable.HashMap.foreach(HashMap.scala:98)
>       at 
> org.apache.samza.checkpoint.OffsetManager.loadDefaults(OffsetManager.scala:378)
>       at 
> org.apache.samza.checkpoint.OffsetManager.start(OffsetManager.scala:175)
>       at 
> org.apache.samza.container.SamzaContainer.startOffsetManager(SamzaContainer.scala:604)
>       at 
> org.apache.samza.container.SamzaContainer.run(SamzaContainer.scala:550)
>       at 
> org.apache.samza.container.SamzaContainer$.safeMain(SamzaContainer.scala:93)
>       at 
> org.apache.samza.container.SamzaContainer$.main(SamzaContainer.scala:67)
>       at org.apache.samza.container.SamzaContainer.main(SamzaContainer.scala)
> {noformat}
> We should add some validation and a meaningful error message for cases when 
> the broadcast inputs don't make sense compared to the metadata. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to