[
http://opencast.jira.com/browse/MH-8148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=27959#comment-27959
]
Josh Holtzman commented on MH-8148:
-----------------------------------
It's important to distinguish workflows from jobs. In Matterhorn 1.0, we
throttled each individual service in its own configuration file, so you could
control exactly how many "inspection" jobs would run at a given time on a given
node. This was very powerful and flexible, but not very easy to configure or
visualize. Every workflow would start, but would eventually block on calls to
the composer services, which were at their maximum number of concurrent jobs
per node.
In 1.1, we switched to a (hopefully) easier to understand and more predictable
scheme: limiting the number of workflow instances that can be started at any
given time. So "max.concurrent" is a configuration made on the admin server
for the workflow service, not for individual worker machines. Worker services
(composer, text analysis, etc.) will try to run as many jobs as you throw at
them. The throttling is done a level up the stack, by the workflow service
instead.
So now, you can tune the entire cluster with a single setting on the admin
server. You may need to experiment with this number to find the concurrency
that balances latency with total throughput.
> max.concurrent doesn't seem to actually set the number of jobs a worker is
> able to do.
> --------------------------------------------------------------------------------------
>
> Key: MH-8148
> URL: http://opencast.jira.com/browse/MH-8148
> Project: Matterhorn Project
> Issue Type: Bug
> Components: Architecture & Services
> Affects Versions: 1.2
> Reporter: Adam McKenzie
> Assignee: Josh Holtzman
> Priority: Critical
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
http://opencast.jira.com/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
_______________________________________________
Matterhorn mailing list
[email protected]
http://lists.opencastproject.org/mailman/listinfo/matterhorn
To unsubscribe please email
[email protected]
_______________________________________________
