Shanthoosh Venkataraman created SAMZA-1554:
----------------------------------------------
Summary: Host affinity in standalone.
Key: SAMZA-1554
URL: https://issues.apache.org/jira/browse/SAMZA-1554
Project: Samza
Issue Type: New Feature
Reporter: Shanthoosh Venkataraman
Assignee: Shanthoosh Venkataraman
Samza framework enables its users to build stateful stream processing
applications–that is, applications that remember information about past events
in a local state(store), which will be then used to influence the processing of
future events from the stream. Local state is a fundamental and enabling
concept in stream processing which is required and essential to support a
majority of common use cases such as stream-stream join, stream-table join,
windowing etc.
Local store of a task instance is backed up by an log compacted kafka topic
referred to as change-log. When a task instance commits, incremental local task
store updates are flushed to the kafka topic. When a task instance runs on a
host that doesn’t have latest local store, it’s restored by replaying messages
from the change-log stream. For large stateful jobs, this restoration phase
takes longer time, thus preventing the application from starting up and
processing events from the input streams. Host affinity is a feature that
maintains stickiness between a task and physical host and offers best-effort
guarantees that a task instance will be assigned to run on the same physical it
had ran before.
This tracks the work required to accomplish this feature.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
