Michael England created DRILL-3675:
--------------------------------------
Summary: Node-label style drillbit ringfencing for Drill queues
for true multi-tenancy
Key: DRILL-3675
URL: https://issues.apache.org/jira/browse/DRILL-3675
Project: Apache Drill
Issue Type: Improvement
Reporter: Michael England
Many of us are used to node labels in YARN which allows jobs to be submitted to
a queue which only executes on a subset of nodes e.g. in a 100 node cluster,
queue 1 may only be allowed to execute jobs on nodes 1-20, queue 2 can execute
on nodes 21-50 and so on. This is necessary in a true multi-tenant environment.
Drill should have the ability to set up queues with node/drillbit labels which
only allows queries submitted to that queue to execute on a defined set of
drillbits. This would allow completely different applications to each have
their own queue and run on a subset of nodes, meaning that resource contention
is avoided. Currently, if multiple applications share a cluster and use drill,
they can cause resource contention and blocking, meaning it is inadequate for a
production multi-tenant environment.
In many mid-sized companies, different application areas will purchase a subset
of nodes to be used as part of a large cluster, and therefore Drill should
provide the option to limit Drill queries to the nodes allocated to them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
