[GitHub] [spark] ScrapCodes opened a new pull request #27520: [SPARK-30771] Avoid failed mount warning from kubernetes and support the optional mount.

Mon, 10 Feb 2020 02:05:58 -0800 

ScrapCodes opened a new pull request #27520: [SPARK-30771] Avoid failed mount 
warning from kubernetes and support the optional mount.
URL: https://github.com/apache/spark/pull/27520
 
 
   
   ### What changes were proposed in this pull request?
   
   This PR changes the way we create kubernetes resources, before this PR, we 
create the driver pod which has reference to various resources including 
mounted volumes and configmaps. After creating the driver pod, we create other 
resources. This PR simply reverses the order, it first creates the configmaps 
and then tries to mount them as k8s volumes.
   
   ```
   kubectl describe po spark-driver
   
   ```
   ### Before this PR
   <img width="1203" alt="Screenshot 2020-02-10 at 3 10 01 PM" 
src="https://user-images.githubusercontent.com/992952/74140116-7d1af200-4c1a-11ea-9180-1db1fb943094.png";>
   
   ### After this PR
   <img width="1203" alt="Screenshot 2020-02-10 at 3 10 16 PM" 
src="https://user-images.githubusercontent.com/992952/74140123-80ae7900-4c1a-11ea-8e6e-12adba1bfd2b.png";>
   
   ### Why are the changes needed?
   
https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.17/#configmapvolumesource-v1-core
   Kubernetes allows an optional field indicating, that if the mount for this 
config map fails, then it is not reattempted nor the pod is declared to be 
failed.
   In our current code base, we try to mount the volumes and create them later, 
it works because, kubernetes reattempts failed mounting attempt, because the 
`optional` field is `false` by default.
   But, if this optional field is set to true, then that mount will not take 
place at all. Because, when the mount is performed the volume is not created - 
so mount fails. And this time the mount is not reattempted because the optional 
field is set as true.
   
   ### Does this PR introduce any user-facing change?
   No
   
   ### How was this patch tested?
   I have tested the patch, by manually running with and without the change and 
observed the event log in kubectl describe.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[email protected]


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to