bkonold opened a new pull request #1331: SAMZA-2495: InMemorySystemAdmin offsets off by one when END_OF_STREAM present URL: https://github.com/apache/samza/pull/1331 I found this when refactoring side input restore to use RunLoop (https://github.com/apache/samza/pull/1330); this patch is necessary to prevent integration tests from failing after that refactor as marking side inputs as caught relies on newest offset. **Symptom:** When an end of stream message is present in an in memory SSP, the new and upcoming offset are reported as 1 higher than they should be. Example: | E1, offset: 0 | E2, offset: 1 | ... | E4, offset: 3 | EndOfStream | In the above case, InMemoryManager will report a newestOffset of 4 and an upcomingOffset of 5; they should be 3 and 4 respectively. **Cause:** InMemoryManager treats end of stream as any other envelope with a numeric offset even though it is not. **Changes:** Check in `InMemoryManager.constructSystemStreamMetadata` whether the last message in the stream is an EndOfStream, and adjust offsets accordingly. **Tests:** Added a unit test to verify that offsets are adjusted by 1 in the case of EndOfStream.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
