[jira] [Updated] (SAMZA-2496) TestContainerHeartbeatMonitor does not properly stop the ContainerHeartbeatMonitor
[ https://issues.apache.org/jira/browse/SAMZA-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cameron Lee updated SAMZA-2496: --- Description: TestContainerHeartbeatMonitor does not properly stop the ContainerHeartbeatMonitor, so after 2 minutes, it can do a force shutdown of the whole test process if the test process is still alive. (was: TestContainerHeartbeatMonitor does not properly stop the ContainerHeartbeatMonitor, so after 2 minutes, it can do a force shutdown of the whole test process if the test process is still alive. The force shutdown task is set up using ScheduledExecutorService.schedule. ScheduledExecutorService.shutdown does not seem to cancel tasks created using the "schedule" method. It seems reasonable to use ScheduledExecutorService.shutdownNow when calling ContainerHeartbeatMonitor.stop, which will also cancel the force shutdown task.) > TestContainerHeartbeatMonitor does not properly stop the > ContainerHeartbeatMonitor > -- > > Key: SAMZA-2496 > URL: https://issues.apache.org/jira/browse/SAMZA-2496 > Project: Samza > Issue Type: Improvement >Reporter: Cameron Lee >Assignee: Cameron Lee >Priority: Major > > TestContainerHeartbeatMonitor does not properly stop the > ContainerHeartbeatMonitor, so after 2 minutes, it can do a force shutdown of > the whole test process if the test process is still alive. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SAMZA-2496) TestContainerHeartbeatMonitor does not properly stop the ContainerHeartbeatMonitor
[ https://issues.apache.org/jira/browse/SAMZA-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cameron Lee updated SAMZA-2496: --- Summary: TestContainerHeartbeatMonitor does not properly stop the ContainerHeartbeatMonitor (was: Clean up ContainerHeartbeatMonitor stop flow) > TestContainerHeartbeatMonitor does not properly stop the > ContainerHeartbeatMonitor > -- > > Key: SAMZA-2496 > URL: https://issues.apache.org/jira/browse/SAMZA-2496 > Project: Samza > Issue Type: Improvement >Reporter: Cameron Lee >Assignee: Cameron Lee >Priority: Major > > TestContainerHeartbeatMonitor does not properly stop the > ContainerHeartbeatMonitor, so after 2 minutes, it can do a force shutdown of > the whole test process if the test process is still alive. > The force shutdown task is set up using ScheduledExecutorService.schedule. > ScheduledExecutorService.shutdown does not seem to cancel tasks created using > the "schedule" method. It seems reasonable to use > ScheduledExecutorService.shutdownNow when calling > ContainerHeartbeatMonitor.stop, which will also cancel the force shutdown > task. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (SAMZA-2496) Clean up ContainerHeartbeatMonitor stop flow
Cameron Lee created SAMZA-2496: -- Summary: Clean up ContainerHeartbeatMonitor stop flow Key: SAMZA-2496 URL: https://issues.apache.org/jira/browse/SAMZA-2496 Project: Samza Issue Type: Improvement Reporter: Cameron Lee Assignee: Cameron Lee TestContainerHeartbeatMonitor does not properly stop the ContainerHeartbeatMonitor, so after 2 minutes, it can do a force shutdown of the whole test process if the test process is still alive. The force shutdown task is set up using ScheduledExecutorService.schedule. ScheduledExecutorService.shutdown does not seem to cancel tasks created using the "schedule" method. It seems reasonable to use ScheduledExecutorService.shutdownNow when calling ContainerHeartbeatMonitor.stop, which will also cancel the force shutdown task. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [samza] kw2542 opened a new pull request #1332: SAMZA-2442: Update doc/tutorials from config factory to config loader
kw2542 opened a new pull request #1332: SAMZA-2442: Update doc/tutorials from config factory to config loader URL: https://github.com/apache/samza/pull/1332 Update documentations and left over scripts from config factory to config loader. API Changes: None Upgrade Instructions: None Usage Instructions: None This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [samza] bkonold closed pull request #1331: SAMZA-2495: InMemorySystemAdmin offsets off by one when END_OF_STREAM present
bkonold closed pull request #1331: SAMZA-2495: InMemorySystemAdmin offsets off by one when END_OF_STREAM present URL: https://github.com/apache/samza/pull/1331 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [samza] bkonold commented on issue #1331: SAMZA-2495: InMemorySystemAdmin offsets off by one when END_OF_STREAM present
bkonold commented on issue #1331: SAMZA-2495: InMemorySystemAdmin offsets off by one when END_OF_STREAM present URL: https://github.com/apache/samza/pull/1331#issuecomment-604634892 This may not be the right fix. Discarding for now while I spend more time investigating how end of stream is used This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [samza] bkonold opened a new pull request #1331: SAMZA-2495: InMemorySystemAdmin offsets off by one when END_OF_STREAM present
bkonold opened a new pull request #1331: SAMZA-2495: InMemorySystemAdmin offsets off by one when END_OF_STREAM present URL: https://github.com/apache/samza/pull/1331 I found this when refactoring side input restore to use RunLoop (https://github.com/apache/samza/pull/1330); this patch is necessary to prevent integration tests from failing after that refactor as marking side inputs as caught relies on newest offset. **Symptom:** When an end of stream message is present in an in memory SSP, the new and upcoming offset are reported as 1 higher than they should be. Example: | E1, offset: 0 | E2, offset: 1 | ... | E4, offset: 3 | EndOfStream | In the above case, InMemoryManager will report a newestOffset of 4 and an upcomingOffset of 5; they should be 3 and 4 respectively. **Cause:** InMemoryManager treats end of stream as any other envelope with a numeric offset even though it is not. **Changes:** Check in `InMemoryManager.constructSystemStreamMetadata` whether the last message in the stream is an EndOfStream, and adjust offsets accordingly. **Tests:** Added a unit test to verify that offsets are adjusted by 1 in the case of EndOfStream. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (SAMZA-2067) Samza on Kubernetes
[ https://issues.apache.org/jira/browse/SAMZA-2067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiqing Yang updated SAMZA-2067: Fix Version/s: (was: 0.15.0) 1.4 > Samza on Kubernetes > --- > > Key: SAMZA-2067 > URL: https://issues.apache.org/jira/browse/SAMZA-2067 > Project: Samza > Issue Type: New Feature >Reporter: Weiqing Yang >Priority: Major > Fix For: 1.4 > > Time Spent: 3h 40m > Remaining Estimate: 0h > > This Jira is to support Samza on Kubernetes. > Please refer to the attached document for more details, including the > background, design, etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (SAMZA-2067) Samza on Kubernetes
[ https://issues.apache.org/jira/browse/SAMZA-2067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiqing Yang updated SAMZA-2067: Fix Version/s: (was: 1.1) > Samza on Kubernetes > --- > > Key: SAMZA-2067 > URL: https://issues.apache.org/jira/browse/SAMZA-2067 > Project: Samza > Issue Type: New Feature >Reporter: Weiqing Yang >Priority: Major > Fix For: 0.15.0 > > Time Spent: 3h 40m > Remaining Estimate: 0h > > This Jira is to support Samza on Kubernetes. > Please refer to the attached document for more details, including the > background, design, etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
