Are connector offsets persisted across worker restarts? I had thought they would be since they are written to the connect-offsets topic. I have code set up to save the offset as a field in my SourceTask, which works fine. I can restart the task and the offsets seem to be persisted, though I don't know where. Does the task stay in memory when restarted and isn't re-instantiated? If so then it would just be reading the offsets from the object itself.
I've checked and the connect-offsets topic has data written to it that seems to match what I'd expect. The problem comes when I restart the worker itself. It seems to start right at the beginning again. I'm using the OffsetStorageReader on context to get the offset at start and at poll. This is a cluster of 1 kafka broker, 1 zookeeper node, and 1 kafka connect worker running in distributed mode. Everything is running on the same VM. Any hints or tips? Anything else I should look into? Thanks, Daniel Wilson-Thomas
