Clément MATHIEU created CRUNCH-665:
--------------------------------------
Summary: Make MRExecutor pollInterval configurable
Key: CRUNCH-665
URL: https://issues.apache.org/jira/browse/CRUNCH-665
Project: Crunch
Issue Type: Improvement
Components: Core
Reporter: Clément MATHIEU
Assignee: Josh Wills
{{MRExecutor#pollInterval}} is currently hard coded to 50..1_000 for local mode
and 500..10_000 otherwise. While being a sensible trade-off, one might want to
poll more aggressively. For example, 1 seconds is usually too high for small
integration tests.
A few experiments on internal test suites showed that lowering poll interval
could halve execution time.
What do you think about adding a new configuration parameter that would
override default config? I can craft a patch if needed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
