Bharath Kumarasubramanian created SAMZA-2593:
------------------------------------------------
Summary: Update task callback to store only necessary fields
instead of entire message envelope
Key: SAMZA-2593
URL: https://issues.apache.org/jira/browse/SAMZA-2593
Project: Samza
Issue Type: Improvement
Reporter: Bharath Kumarasubramanian
Assignee: Bharath Kumarasubramanian
TaskCallbackImpl currently stores the entire message envelope. In case of jobs
that have 1000s of pending callbacks, this adds up to significant memory
pressure. The only use for message envelope is to trace the SSP information and
the offset associated with it so that we commit the right offset during our
commit cycle.
*Suggested solution*: Modify TaskCallbackImpl to only store SSP and Offset
information.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
