Re: can't apply mappartitions to dataframe generated from carboncontext

modify carboncontext.scala file adding @transient to two member variables and problem solved @transient val sc: SparkContext, @transient val hiveClientInterface = metadataHive - FFCS研究院 -- View this message in context:

Re: can't apply mappartitions to dataframe generated from carboncontext

Hi can you share me your test steps for reproducing this issue ? I mean completed test steps. Thanks. Chenerlu. -- View this message in context:

Re: can't apply mappartitions to dataframe generated from carboncontext

org.apache.spark.SparkException: Task not serializable at org.apache.spark.util.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:304) at org.apache.spark.util.ClosureCleaner$.org$apache$spark$util$ClosureCleaner$$clean(ClosureCleaner.scala:294) at

Re: can't apply mappartitions to dataframe generated from carboncontext

Hi, Mic sun Can you ping your error message directly ? It seems I can't get access to your appendix. Thanks in advance. Regards. Chenerlu. -- View this message in context: