On a related note, one bug we hit when using Samza was not populating
all the partitions in our test environment for a bootstrap stream. That
happened
because we partition by something we call an application ID and we were
testing
using only one application's data. The job would catch up to the bootstrap
stream
and then freeze. The solution was to add more data from other applications.
I know this was kind of by design, but it seems like a behavior that is
bound
to cause many problems for others as well. I would like to see this change.
Lukas
-----Original Message-----
From: Roger Hoover
Sent: Sunday, January 4, 2015 3:31 PM
To: [email protected]
Subject: Samza 0.8.0 - Job hangs if bootstrap topic is empty
Hi,
If someone forgets to populate a bootstrap topic before starting a job, the
job will hang indefinitely without processing any non-bootstrap messages,
even if the bootstrap topic later populated. This is unexpected and hard
for operations to debug.
I think Samza should either treat empty bootstrap topics as already caught
up or as fatal errors. I created a JIRA here:
https://issues.apache.org/jira/browse/SAMZA-502
Thanks,
Roger
