[
https://issues.apache.org/jira/browse/SAMZA-1017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prateek Maheshwari updated SAMZA-1017:
--------------------------------------
Attachment: throttling-scheduler.patch
Added disk quota based throttling to AsyncRunLoop.
This patch adds a ThrottlingScheduler that schedules callbacks with a delay
proportional to original work duration and current work factor. Also adds a
Throttleable interface, implemented by RunLoop, AsyncRunLoop,
ThrottlingExecutor and ThrottlingScheduler.
When AsyncRunLoop is throttled, it delays the onComplete() callback from
processAsync() using ThrottlingScheduler.
Notes:
1. Adding additional delay to process()/processAsync() callback will not
throttle the run loop if task processing rate > message throughput. E.g., a
stream with process() time significantly lower than message inter-arrival time.
2. If throttled, users can increase their throughput back to original by
increasing task.max.concurrency and redeploying their jobs.
> Disk Quotas - Add throttler for AsyncRunLoop
> --------------------------------------------
>
> Key: SAMZA-1017
> URL: https://issues.apache.org/jira/browse/SAMZA-1017
> Project: Samza
> Issue Type: New Feature
> Reporter: Prateek Maheshwari
> Assignee: Prateek Maheshwari
> Fix For: 0.11.0
>
> Attachments: throttling-scheduler.patch
>
>
> Continuation of Disk Quotas work started with SAMZA-924. In this patch we add
> support for throttling async run loop execution.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
