[
https://issues.apache.org/jira/browse/DRILL-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16426218#comment-16426218
]
ASF GitHub Bot commented on DRILL-143:
--------------------------------------
Github user kkhatua commented on a diff in the pull request:
https://github.com/apache/drill/pull/1200#discussion_r179292376
--- Diff: distribution/src/resources/drillbit.sh ---
@@ -127,6 +127,44 @@ check_before_start()
fi
}
+check_after_start(){
+ #check if the process is running
+ if [ -f $pid ]; then
+ dbitProc=$(ps -ef | grep `cat $pid` | grep Drillbit)
+ if [ -n "$dbitProc" ]; then
+ # Check and enforce for CGroup
+ if [ -n "$DRILLBIT_CGROUP" ]; then
+ check_and_enforce_cgroup `cat $pid`
+ fi
+ fi
+ fi
+}
+
+check_and_enforce_cgroup(){
+ dbitPid=$1;
+ #if [ $(`ps -o cgroup` | grep -c $DRILLBIT_CGROUP ) -eq 1 ]; then
+ if [ -f /cgroup/cpu/${DRILLBIT_CGROUP}/cgroup.procs ]; then
+ echo $dbitPid > /cgroup/cpu/${DRILLBIT_CGROUP}/cgroup.procs
+ # Verify Enforcement
+ cgroupStatus=`grep -w $pid
/cgroup/cpu/${DRILLBIT_CGROUP}/cgroup.procs`
+ if [ -z "$cgroupStatus" ]; then
+ #Ref:
https://www.kernel.org/doc/Documentation/scheduler/sched-bwc.txt
+ let cpu_quota=`cat /cgroup/cpu/${DRILLBIT_CGROUP}/cpu.cfs_quota_us`
+ let cpu_period=`cat
/cgroup/cpu/${DRILLBIT_CGROUP}/cpu.cfs_period_us`
+ if [ $cpu_period -gt 0 ] && [ $cpu_quota -gt 0 ]; then
+ coresAllowed="(up to "`echo $(( 100 * $cpu_quota / $cpu_period
)) | sed 's/..$/.&/'`" cores allowed)"
+ fi
+ echo "WARN: Drillbit's CPU resource usage will be managed under
the CGroup : $DRILLBIT_CGROUP "$coresAllowed
+ else
+ echo "ERROR: Failed to add Drillbit to CGroup ( $DRILLBIT_CGROUP )
for resource usage management. Ensure that the cgroup manages CPU"
+ exit 1
+ fi
+ else
+ echo "ERROR: cgroup $DRILLBIT_CGROUP does not found. Ensure that
daemon is running and cgroup exists"
+ exit 1
--- End diff --
Good point. Ideally, we should prevent the Drillbit from starting up (or in
this case, it should shut down), if CGroups couldn't be applied. My concern is
that if CGroups is not being enforced, we're running a process that can
(potentially) consume excess CPU resources.
Should I shut down the Drillbit in such a scenario, or move on with just a
WARN message?
> Support CGROUPs resource management
> -----------------------------------
>
> Key: DRILL-143
> URL: https://issues.apache.org/jira/browse/DRILL-143
> Project: Apache Drill
> Issue Type: New Feature
> Reporter: Jacques Nadeau
> Assignee: Kunal Khatua
> Priority: Major
> Fix For: 1.14.0
>
> Attachments: 253ce178-ddeb-e482-cd64-44ab7284ad1c.sys.drill
>
>
> For the purpose of playing nice on clusters that don't have YARN, we should
> write up configuration and scripts to allows users to run Drill next to
> existing workloads without sharing resources.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
