[
https://issues.apache.org/jira/browse/SAMZA-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15497448#comment-15497448
]
Yi Pan (Data Infrastructure) commented on SAMZA-1018:
-----------------------------------------------------
[~twbecker], just to check: do you have topic auto-creation feature turned on
in your Kafka cluster?
> Cannot consume non-existent topic
> ---------------------------------
>
> Key: SAMZA-1018
> URL: https://issues.apache.org/jira/browse/SAMZA-1018
> Project: Samza
> Issue Type: Bug
> Components: container
> Affects Versions: 0.10.1
> Environment: Kafka 0.10.0.0
> Reporter: Tommy Becker
>
> When starting a stream job that consumes a topic that does not yet exist, the
> job dies with the following exception:
> {noformat}
> Exception in thread "main" java.lang.IllegalArgumentException: No tasks
> found. Likely due to no input partitions. Can't run a job with no tasks.
> at
> org.apache.samza.container.grouper.task.GroupByContainerCount.validateTasks(GroupByContainerCount.java:193)
> at
> org.apache.samza.container.grouper.task.GroupByContainerCount.balance(GroupByContainerCount.java:86)
> at
> org.apache.samza.coordinator.JobModelManager$.refreshJobModel(JobCoordinator.scala:278)
> at
> org.apache.samza.coordinator.JobModelManager$.jobModelGenerator$1(JobCoordinator.scala:211)
> at
> org.apache.samza.coordinator.JobModelManager$.initializeJobModel(JobCoordinator.scala:217)
> at
> org.apache.samza.coordinator.JobModelManager$.getJobCoordinator(JobCoordinator.scala:122)
> at
> org.apache.samza.coordinator.JobModelManager$.apply(JobCoordinator.scala:106)
> at
> org.apache.samza.coordinator.JobModelManager$.apply(JobCoordinator.scala:112)
> at
> org.apache.samza.job.local.ThreadJobFactory.getJob(ThreadJobFactory.scala:40)
> at org.apache.samza.job.JobRunner.run(JobRunner.scala:129)
> at org.apache.samza.job.JobRunner$.main(JobRunner.scala:66)
> at org.apache.samza.job.JobRunner.main(JobRunner.scala)
> {noformat}
> This seems to be caused by the fix for SAMZA-971, specifically passing
> partitionsMetadataOnly = true to the StreamMetadataCache in
> JobModelManager#getInputStreamPartitions
> https://github.com/apache/samza/commit/920f803a2e3dab809f4d7bb518259b0f4164407f
> Note the input topic is subsequently created, so a restart of the job will
> likely succeed. Being able to consume topics which have not yet been created
> is nice to avoid imposing a startup order between the jobs and the processes
> which produce to the topics. Setting partitionsMetadataOnly to back to false
> fixed the issue for us, but I'm not sure if this is the best fix. For the
> record, we are using Kafka 0.10.0.0
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
