[
https://issues.apache.org/jira/browse/SAMZA-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lakshmi Manasa Gaduputi updated SAMZA-2689:
-------------------------------------------
Description:
# For a job with elasticity factor = X, each input SystemStreamPartition is
split into X parts aka SystemStreamPartitionKeyHashes. ElasticJobModel has
elastic task models where each original task ( in the original JobModel) now
has X elastic tasks each of which consumes an SSPKh pertaining to the SSP of
the original task.
# Create an ElasticJobModel and thus assign elastic tasks to containers. To
achieve this, the groupers which group the input SSPs into task models, need to
be updated.
# ElasticJobModel created in the earlier sub-task creates X elastic tasks for
each original task where elasticity.factor=X. Each elastic task consumes an
SSPKh = 1/X th of the input SSP. This task is to ensure each elastic task
actually processes only the messages within that SSPKh.
was:# For a job with elasticity factor = X, each input SystemStreamPartition
is split into X parts aka SystemStreamPartitionKeyHashes. ElasticJobModel has
elastic task models where each original task ( in the original JobModel) now
has X elastic tasks each of which consumes an SSPKh pertaining to the SSP of
the original task.
> [Elasticity] introduce ElasticJobModel which has SystemStreamPartitionKeyHash
> -----------------------------------------------------------------------------
>
> Key: SAMZA-2689
> URL: https://issues.apache.org/jira/browse/SAMZA-2689
> Project: Samza
> Issue Type: Sub-task
> Reporter: Lakshmi Manasa Gaduputi
> Assignee: Lakshmi Manasa Gaduputi
> Priority: Major
>
> # For a job with elasticity factor = X, each input SystemStreamPartition is
> split into X parts aka SystemStreamPartitionKeyHashes. ElasticJobModel has
> elastic task models where each original task ( in the original JobModel) now
> has X elastic tasks each of which consumes an SSPKh pertaining to the SSP of
> the original task.
> # Create an ElasticJobModel and thus assign elastic tasks to containers. To
> achieve this, the groupers which group the input SSPs into task models, need
> to be updated.
> # ElasticJobModel created in the earlier sub-task creates X elastic tasks
> for each original task where elasticity.factor=X. Each elastic task consumes
> an SSPKh = 1/X th of the input SSP. This task is to ensure each elastic task
> actually processes only the messages within that SSPKh.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
