Hi Team, I am trying to create a connector for carbondata. Currently i am working on creating OutputFormat for carbondata. For that i need to create a carboncontext which i initialise in configure method. Internally CarbonContext uses sparkContext.
I am able to create carbonContext through my code but when it run statement "executionEnvironment.execute() Exception in thread "main" org.apache.flink.optimizer.CompilerException: Error translating node 'Data Sink "org.apache.flink.CarbonDataOutputFormat@76225435" : NONE [[ GlobalProperties [partitioning=RANDOM_PARTITIONED] ]] [[ LocalProperties [ordering=null, grouped=null, unique=null] ]]': Could not write the user code wrapper class org.apache.flink.api.common.operators.util.UserCodeObjectWrapper : java.io.NotSerializableException: org.apache.spark.sql.hive.client.ClientWrapper at org.apache.flink.optimizer.plantranslate.JobGraphGenerator.preVisit(JobGraphGenerator.java:381) at org.apache.flink.optimizer.plantranslate.JobGraphGenerator.preVisit(JobGraphGenerator.java:106) at org.apache.flink.optimizer.plan.SingleInputPlanNode.accept(SingleInputPlanNode.java:198) at org.apache.flink.optimizer.plan.OptimizedPlan.accept(OptimizedPlan.java:128) at org.apache.flink.optimizer.plantranslate.JobGraphGenerator.compileJobGraph(JobGraphGenerator.java:192) at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:188) at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91) at org.apache.flink.api.java.ExecutionEnvironment.execute(ExecutionEnvironment.java:926) at CarbonJDBCExample.main(CarbonJDBCExample.java:86) Caused by: org.apache.flink.runtime.operators.util.CorruptConfigurationException: Could not write the user code wrapper class org.apache.flink.api.common.operators.util.UserCodeObjectWrapper : java.io.NotSerializableException: org.apache.spark.sql.hive.client.ClientWrapper at org.apache.flink.runtime.operators.util.TaskConfig.setStubWrapper(TaskConfig.java:281) at org.apache.flink.optimizer.plantranslate.JobGraphGenerator.createDataSinkVertex(JobGraphGenerator.java:903) at org.apache.flink.optimizer.plantranslate.JobGraphGenerator.preVisit(JobGraphGenerator.java:278) ... 8 more Caused by: java.io.NotSerializableException: org.apache.spark.sql.hive.client.ClientWrapper at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) at org.apache.flink.util.InstantiationUtil.serializeObject(InstantiationUtil.java:317) at org.apache.flink.util.InstantiationUtil.writeObjectToConfig(InstantiationUtil.java:254) at org.apache.flink.runtime.operators.util.TaskConfig.setStubWrapper(TaskConfig.java:279) ... 10 more -- Warm Regards, Sangeeta Gulia Software Consultant Knoldus Software LLP
