chenyulin0719 commented on code in PR #422: URL: https://github.com/apache/yunikorn-site/pull/422#discussion_r1571859465
########## docs/user_guide/gang_scheduling.md: ########## @@ -216,10 +221,18 @@ See the [Spark documentation](https://spark.apache.org/docs/latest/configuration For all the executor pods, ```yaml -Annotations: - # the taskGroup name should match to the names - # defined in the taskGroups field - yunikorn.apache.org/taskGroupName: “spark-executor” +apiVersion: v1 +kind: Pod +metadata: + labels: + spark-app-id: spark-executor + queue: root.sandbox + annotations: + # the taskGroup name should match to the names + # defined in the taskGroups field + yunikorn.apache.org/task-group-name: "spark-executor" Review Comment: Ditto, I think we could just keep this 4 these here. ########## docs/user_guide/gang_scheduling.md: ########## @@ -180,32 +180,37 @@ Each Spark job runs 2 types of pods, driver and executor. Hence, we need to defi The annotations for the driver pod looks like: ```yaml -Annotations: - yunikorn.apache.org/schedulingPolicyParameters: “placeholderTimeoutSeconds=30” - yunikorn.apache.org/taskGroupName: “spark-driver” - yunikorn.apache.org/taskGroup: “ - TaskGroups: [ - { - Name: “spark-driver”, - minMember: 1, - minResource: { - Cpu: 1, - Memory: 2Gi - }, - Node-selector: ..., - Tolerations: ..., - Affinity: ... - }, +apiVersion: v1 +kind: Pod +metadata: + labels: + spark-app-id: spark-driver + queue: root.sandbox + annotations: + yunikorn.apache.org/schedulingPolicyParameters: "placeholderTimeoutInSeconds=30 gangSchedulingStyle=Hard" + yunikorn.apache.org/task-group-name: "spark-driver" + yunikorn.apache.org/task-groups: |- + [{ + "name": "spark-driver", + "minMember": 1, + "minResource": { + "cpu": "1", + "memory": "2Gi" + }, + "nodeSelector": {}, + "tolerations": [], + "affinity": {} + }, { - Name: “spark-executor”, - minMember: 10, - minResource: { - Cpu: 1, - Memory: 2Gi - } - } - ] - ” + "name": "spark-executor", + "minMember": 1, + "minResource": { + "cpu": "1", + "memory": "2Gi" + } + }] +spec: + schedulerName: yunikorn Review Comment: The original idea of this Jira is to fix the wrong gang scheduling annotation name. I'm good with changing the pseudo yaml to be the more realistic expression. However, I suggest retaining 'annotations' and remove apiVersion, kind, metadata.labels, and spec, since it still won't be an executable YAML. And the previous example simply aims to illustrate what 'the annotations for the driver pod' should look like. <img width="884" alt="image" src="https://github.com/apache/yunikorn-site/assets/26764036/876c3c65-4578-4328-b292-5a08e8d3c893";> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
