[
https://issues.apache.org/jira/browse/SAMZA-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16558916#comment-16558916
]
Shanthoosh Venkataraman commented on SAMZA-1786:
------------------------------------------------
Documenting the tests done for validating the patch changes in yarn:
1. Added unit tests for new classes that were added(Fixed the existing unit
tests in LocalityManager/TaskAssignmentManager).
2. All these changes are validated with test jobs present in
samza-hello-samza(https://github.com/apache/samza-hello-samza).
3. LinkedIn production job(maes-tests-host-affinity) is verified with these
changes to validate if things work end-to-end properly.
> Introduce metadata store abstraction.
> -------------------------------------
>
> Key: SAMZA-1786
> URL: https://issues.apache.org/jira/browse/SAMZA-1786
> Project: Samza
> Issue Type: New Feature
> Reporter: Shanthoosh Venkataraman
> Assignee: Shanthoosh Venkataraman
> Priority: Major
>
> As a part of SEP-11, this JIRA introduces MetadataStore interface to store
> task and container locality for both yarn and standalone deployment models.
> Refer to SEP-11 design doc for more details.
>
> Few important points to note:
> # As a part of this changes, LocalityManager/TaskAsssignemtnManager alone
> will be updated to use this interface(subsequently in upcoming future RB's
> other util classes will be moved to use this interface as well).
> # In an immediate followup RB, ZkMetadataStore(storing metadata information
> in zookeeper) will be added. It will be used in standalone to read/write
> locality(through LocalityManager & other standard util classes).
> # In future, executionPlan, other job related metadata(with respect to
> running containers, stream graph) can be stored in any store through the same
> abstraction. In future, if we want to deploy standalone jobs in azure, only
> we need to implement this interface(CosmosDBMetadataStore to store samza
> metadata in cosmos).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
