Pushpendra Jaiswal created FLINK-4331:
-----------------------------------------
Summary: Flink is not able to serialize scala classes / Task Not
Serializable
Key: FLINK-4331
URL: https://issues.apache.org/jira/browse/FLINK-4331
Project: Flink
Issue Type: Bug
Components: DataStream API
Affects Versions: 1.1.0
Reporter: Pushpendra Jaiswal
I have scala class having 2 fields which are vals but flink is saying it
doesn't have setters. Thus task is not serializable.
I tried setters using var but then it says duplicate setter. vals are public
then why it is asking for setters. Flink version 1.1.0
class Impression(val map: Map[String, String],val keySet:Set[String])
==========================================================================
val preAggregate = stream
.filter(impression => {
true
})
.map(impression => {
val xmap = impression.map
val values = valFunction(xmap)
new ImpressionRecord(impression, values._1, values._2, values._3)
})
class Impression does not contain a setter for field map 19:54:49.995 [main]
INFO o.a.f.a.java.typeutils.TypeExtractor - class Impression is not a valid
POJO type 19:54:49.997 [main] DEBUG o.a.flink.api.scala.ClosureCleaner$ -
accessedFields: Map(class -> Set()) Exception in thread "main"
org.apache.flink.api.common.InvalidProgramException: Task not serializable at
org.apache.flink.api.scala.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:172)
at ) Caused by: java.io.NotSerializableException:
org.apache.flink.streaming.api.scala.DataStream at
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1183) at
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547) at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547) at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at
java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) at
org.apache.flink.util.InstantiationUtil.serializeObject(InstantiationUtil.java:301)
at
org.apache.flink.api.scala.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:170)
... 18 more
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
