Dharmesh Kakadia created SPARK-24894:
----------------------------------------
Summary: Invalid DNS name due to hostname truncation
Key: SPARK-24894
URL: https://issues.apache.org/jira/browse/SPARK-24894
Project: Spark
Issue Type: Bug
Components: Kubernetes
Affects Versions: 2.3.1
Reporter: Dharmesh Kakadia
The truncation for hostname happening here
[https://github.com/apache/spark/blob/5ff1b9ba1983d5601add62aef64a3e87d07050eb/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicExecutorFeatureStep.scala#L77]
is a problematic and can lead to DNS names starting with "-".
Originally filled here :
https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/issues/229
```
{{2018-07-23 21:21:42 ERROR Utils:91 - Uncaught exception in thread
kubernetes-pod-allocator
io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST
at: https://kubernetes.default.svc/api/v1/namespaces/default/pods. Message: Pod
"user-archetypes-all-weekly-1532380861251850404-1532380862321-exec-9" is
invalid: spec.hostname: Invalid value:
"-archetypes-all-weekly-1532380861251850404-1532380862321-exec-9": a DNS-1123
label must consist of lower case alphanumeric characters or '-', and must start
and end with an alphanumeric character (e.g. 'my-name', or '123-abc', regex
used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?'). Received status:
Status(apiVersion=v1, code=422,
details=StatusDetails(causes=[StatusCause(field=spec.hostname, message=Invalid
value: "-archetypes-all-weekly-1532380861251850404-1532380862321-exec-9": a
DNS-1123 label must consist of lower case alphanumeric characters or '-', and
must start and end with an alphanumeric character (e.g. 'my-name', or
'123-abc', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?'),
reason=FieldValueInvalid, additionalProperties={})], group=null, kind=Pod,
name=user-archetypes-all-weekly-1532380861251850404-1532380862321-exec-9,
retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status,
message=Pod
"user-archetypes-all-weekly-1532380861251850404-1532380862321-exec-9" is
invalid: spec.hostname: Invalid value:
"-archetypes-all-weekly-1532380861251850404-1532380862321-exec-9": a DNS-1123
label must consist of lower case alphanumeric characters or '-', and must start
and end with an alphanumeric character (e.g. 'my-name', or '123-abc', regex
used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?'),
metadata=ListMeta(resourceVersion=null, selfLink=null,
additionalProperties={}), reason=Invalid, status=Failure,
additionalProperties={}). at
io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:470)
at
io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:409)
at
io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:379)
at
io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:343)
at
io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleCreate(OperationSupport.java:226)
at
io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleCreate(BaseOperation.java:769)
at
io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:356)
at
org.apache.spark.scheduler.cluster.k8s.KubernetesClusterSchedulerBackend$$anon$1$$anonfun$3$$anonfun$apply$3.apply(KubernetesClusterSchedulerBackend.scala:140)
at
org.apache.spark.scheduler.cluster.k8s.KubernetesClusterSchedulerBackend$$anon$1$$anonfun$3$$anonfun$apply$3.apply(KubernetesClusterSchedulerBackend.scala:140)
at org.apache.spark.util.Utils$.tryLog(Utils.scala:1922) at
org.apache.spark.scheduler.cluster.k8s.KubernetesClusterSchedulerBackend$$anon$1$$anonfun$3.apply(KubernetesClusterSchedulerBackend.scala:139)
at
org.apache.spark.scheduler.cluster.k8s.KubernetesClusterSchedulerBackend$$anon$1$$anonfun$3.apply(KubernetesClusterSchedulerBackend.scala:138)
at
scala.collection.MapLike$MappedValues$$anonfun$foreach$3.apply(MapLike.scala:245)
at
scala.collection.MapLike$MappedValues$$anonfun$foreach$3.apply(MapLike.scala:245)
at
scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733)
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40) at
scala.collection.mutable.HashMap.foreach(HashMap.scala:99) at
scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:732)
at scala.collection.MapLike$MappedValues.foreach(MapLike.scala:245) at
scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at
scala.collection.AbstractTraversable.map(Traversable.scala:104) at
org.apache.spark.scheduler.cluster.k8s.KubernetesClusterSchedulerBackend$$anon$1.run(KubernetesClusterSchedulerBackend.scala:145)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)}}
```
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
