Checkpoint and recomputation
Hi dear devs, I recently came across checkpoint functionality in Spark and found (a little surprising) that checkpoint causes the DataFrame to be computed twice unless cache is called before checkpoint. My guess is that this is probably hard to fix and/or maybe checkpoint feature is not very frequently used?
Question about spark on k8s
Hello, devs. In our scenario, we run spark on Kata-like containers, and found the code had written the Kube-DNS domain. If Kube-DNS is not configured in environment, tasks would run failed. My question is, why we wrote the domain name of Kube-DNS in the code? Isn't it better to read domain name from the service? -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
