merrily01 opened a new pull request #25920: [SPARK-29233][KUBERNETES] Add regex expression checks for executorEnv… URL: https://github.com/apache/spark/pull/25920 ### What changes were proposed in this pull request? In kubernetes, there are some naming regular expression requirements and restrictions on environment variable names, such as: - In kubernetes version release-1.7 and earlier, the naming rules of pod environment variable names should meet the requirements of regular expressions: [[A-Za-z_] [A-Za-z0-9_]*](https://github.com/kubernetes/kubernetes/blob/release-1.7/staging/src/k8s.io/apimachinery/pkg/util/validation/validation.go#L169) - In kubernetes version release-1.8 and later, the naming rules of pod environment variable names should meet the requirements of regular expressions: [[-. _ A-ZA-Z][-. _ A-ZA-Z0-9].*](https://github.com/kubernetes/kubernetes/blob/release-1.8/staging/src/k8s.io/apimachinery/pkg/util/validation/validation.go#L305) However, in spark on k8s mode, spark should add restrictions on environmental variable names when creating executorEnv. In addition, we need to use regular expressions adapted to the high version of k8s to increase the restrictions on the names of environmental variables. Otherwise, the pod will not be created properly and the spark application will be suspended. To solve the problem above, a regular validation to executorEnv is added and committed. ### Why are the changes needed? If no validation rules are added, the environment variable names that don't meet the requirements will cause the pod to not be created properly and the application will be suspended. ### Does this PR introduce any user-facing change? No. ### How was this patch tested? Add unit tests and manually run.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
