How to join dstream and JDBCRDD with checkpointing enabled

Hi All, We have a spark streaming job with checkpoint enabled, it executes correctly first time, but throw below exception when restarted from checkpoint. org.apache.spark.SparkException: RDD transformations and actions can only be invoked by the driver, not inside of other transformations; for e

Window Functions with SQLContext

Hi All, As per SPARK-11001 , Window functions should be supported by SQLContext. But when i try to run SQLContext sqlContext = new SQLContext(jsc); WindowSpec w = Window.partitionBy("assetId").orderBy("assetId"); DataFrame df_2 = df1.withColu