[
https://issues.apache.org/jira/browse/SAMZA-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14115600#comment-14115600
]
Chris Riccomini commented on SAMZA-354:
---------------------------------------
bq. This seems like a lot of work for marginal benefit.
It seems like less work to me, actually. It should be easier to just let the
consumer run forever, rather than have to pay attention to the incoming
offsets, and shutdown when current offset == latest offset.
Part of this is just who you envision running the migration tool. If you just
expect the devs to do it themselves, then either way should work. If you expect
ops guys to do it, then having the continual loader run seems preferable, since
it requires less coordination. Another thing to consider is that this tool is
going to have to be run in prod. I don't think we want devs SSH'ing into
production clusters and running checkpoint tool commands. We could obviously
build some wrapper tool, that would run the migration for them, but at that
point, it seems like way less work to just have the tool as I described.
> Write tool to convert old-style checkpoint log to post-SAMZA-123 format
> -----------------------------------------------------------------------
>
> Key: SAMZA-354
> URL: https://issues.apache.org/jira/browse/SAMZA-354
> Project: Samza
> Issue Type: Task
> Affects Versions: 0.8.0
> Reporter: Jakob Homan
> Assignee: David Chen
>
> After SAMZA-123, the checkpoint log has a new format (keyed entries
> interspersed with statelog-partition mapping) and a new name. It would be
> simple to write a tool that would consume an old-style log and write out a
> new-style log, using the GroupByPartition strategy. This would allow
> existing jobs to not lose checkpointing.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
