kondziolka9ld created SPARK-45249:
-------------------------------------
Summary: Why isCheckpointed returns false while as checkpointing
is eager by default?
Key: SPARK-45249
URL: https://issues.apache.org/jira/browse/SPARK-45249
Project: Spark
Issue Type: Question
Components: Spark Core
Affects Versions: 3.3.0
Reporter: kondziolka9ld
Hi!
I consider why `isCheckpointed` method return `false` - could someone explain
it? I would expect `true`.
{code:java}
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 3.3.0
/_/
Using Scala version 2.12.15 (OpenJDK 64-Bit Server VM, Java 11.0.13)
Type in expressions to have them evaluated.
Type :help for more information.scala>
sc.setCheckpointDir("file:///tmp/")scala> val df1 = Seq(1,2,3,4).toDF
df1: org.apache.spark.sql.DataFrame = [value: int]scala> val df2 =
df1.checkpoint()
df2: org.apache.spark.sql.Dataset[org.apache.spark.sql.Row] = [value: int]
scala> df2.rdd.isCheckpointed
res2: Boolean = false // why false?{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
