[
https://issues.apache.org/jira/browse/SAMZA-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15988368#comment-15988368
]
ASF GitHub Bot commented on SAMZA-1212:
---------------------------------------
GitHub user navina opened a pull request:
https://github.com/apache/samza/pull/148
SAMZA-1212 - Refactor interaction between StreamProcessor, JobCoordinator
and SamzaContainer
See SAMZA-1212 for motivation toward this refactoring.
Changes here are:
* Removed awaitStart (blocking) method in StreamProcessor, JobCoordinator
and SamzaContainer
* Introduced SamzaContainerListener and JobCoordinatorListener interface
implemented by StreamProcessor
* Introduced SamzaContainerStatus to handler failures and lifecycle using
Listener interfaces
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/navina/samza SAMZA-1212
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/samza/pull/148.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #148
----
commit d81cfdca4b84f02b86ab70657c8c4636e8902b9a
Author: Navina Ramesh <[email protected]>
Date: 2017-04-14T00:20:38Z
Remove containerId from SamzaContainer.apply
commit c4a10242b6e85345ed4515b98ec407435c1fdce1
Author: Navina Ramesh <[email protected]>
Date: 2017-04-14T00:22:01Z
Removing onBecomingLeader for readability
commit 56028361552b37a27991c6cac1f3e00cc3d3a0f2
Author: Navina Ramesh <[email protected]>
Date: 2017-04-14T00:26:01Z
Removing awaitStart
commit fd99fd65fb437afed04240b3971b3cefc1f52f1d
Author: Navina Ramesh <[email protected]>
Date: 2017-04-14T19:12:46Z
Added JobCoordinator Listener. Trying to remove Samzacontainer controller
commit e77aa502df74cedb87c50a8e039135975504381e
Author: Navina Ramesh <[email protected]>
Date: 2017-04-25T01:36:14Z
Adding ProcessorErrorHandler, SamzaContainerStatus, JobCoordinatorListener,
SamzaContainerListener
commit 3cbf259c1e9fea7a4d24af93a812e75d9947aac8
Author: Navina Ramesh <[email protected]>
Date: 2017-04-26T23:01:24Z
Documenting state transitions for SamzaContainer
commit 679b2f54aa7a39c8dae688f9b446aa9bad9d267f
Author: Navina Ramesh <[email protected]>
Date: 2017-04-26T23:03:00Z
adding some log lines in LocalContainerRunner
commit 3b65cc983d7d734d6fdf2a81cb155fbad0e774b3
Author: Navina Ramesh <[email protected]>
Date: 2017-04-27T01:36:25Z
Fixed integration test failures by throwing the exception in the listener
for ThreadJob
commit b1b61f58b2e06a1e7f5fc602fe9007d4c1a003a0
Author: Navina Ramesh <[email protected]>
Date: 2017-04-27T19:42:54Z
Added a few tests in TestSamzaContainer
commit a2db96924ebd479e2110fc611c86c3c310336212
Author: Navina Ramesh <[email protected]>
Date: 2017-04-27T23:38:29Z
Added test for sp.stop()
commit bc74cd5670aacfe5c4eae7968973e68f9f700876
Author: Navina Ramesh <[email protected]>
Date: 2017-04-28T00:23:32Z
Adding setContainerListener explicitly in SamzaContainer
commit 07adf3c6ce39a893a0995498bc012cf6c14c43be
Author: Navina Ramesh <[email protected]>
Date: 2017-04-28T02:38:24Z
Added documentation in JobCoordinator interface
commit 78a73540cc0cd84db286737b190c596dcde93d1f
Author: Navina Ramesh <[email protected]>
Date: 2017-04-28T02:43:00Z
Removed ProcessorErrorHandler
commit 5d1b28c6b566ca691a955d94bb1daf29a96737ef
Author: Navina Ramesh <[email protected]>
Date: 2017-04-28T02:45:47Z
Removing commetned out code
commit 42ffc7d6c1d5e657b35d9482df30f0e201bdbb27
Author: Navina Ramesh <[email protected]>
Date: 2017-04-28T05:00:40Z
Adding docs to JobCoordinatorListener
commit 5ff163cf19c85875a0e2a8d85682487186ffc6c5
Author: Navina Ramesh <[email protected]>
Date: 2017-04-28T05:36:02Z
Added javadocs for SamzaContainerListener
commit f3551656037a058aebd62e9f7dacaafeb49d2f94
Author: Navina Ramesh <[email protected]>
Date: 2017-04-28T06:33:49Z
Cleaning up StreamProcessor code and jobCoordinator docs
commit c624d75afd77dd028a4406d6e07d2ef801098b03
Author: Navina Ramesh <[email protected]>
Date: 2017-04-28T07:09:36Z
Fixing standaloneJobCoordinator
commit c116a3c55149a4cca738a66ec925569385568be9
Author: Navina Ramesh <[email protected]>
Date: 2017-04-28T07:14:04Z
Adding null checks on processorListener
commit 6f0715c4944255409bd78fd178c8e9976e60f485
Author: Navina Ramesh <[email protected]>
Date: 2017-04-28T07:32:37Z
Fixing ZkJobCoordinator
----
> Refactor interaction between StreamProcessor, JobCoordinator and
> SamzaContainer
> -------------------------------------------------------------------------------
>
> Key: SAMZA-1212
> URL: https://issues.apache.org/jira/browse/SAMZA-1212
> Project: Samza
> Issue Type: Bug
> Reporter: Navina Ramesh
> Assignee: Navina Ramesh
> Fix For: 0.13.0
>
>
> Currently, StreamProcessor is a dummy wrapper for JobCoordinator and the
> JobCoordinator directly controls the lifecycle of SamzaContainer. This has a
> couple of drawbacks:
> 1. Each implementation of JobCoordinator needs to manage the container
> lifecycle. This seems redundant as the lifecycle of SamzaContainer is driven
> by availability of JobModel and updates to it.
> 2. The StreamProcessor interface requirements kind of bleed into
> JobCoordinator interface. For example, StreamProcessor interacts with
> user-api. If the requirements for the user-api change , it will directly
> impact the JobCoordinator interface as well. A good example of such a design
> creep is the "awaitStop" interface. While this is a reasonable requirement
> from the user's perspective of a StreamProcessor, it should not impose the
> same requirement on every implementation of JobCoordinator.
> 3. Error handling is pretty straight-forward when there is a control
> hierarchy of components. However, some implementations of JobCoordinators are
> synchronous, while others are Asynchronous. This will directly impact how
> errors/exceptions are propagated across the stack. It will be much cleaner if
> the StreamProcessor can handle the errors/exception from both container and
> coordinator.
> These are the primary motivations for refactoring.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
