[
https://issues.apache.org/jira/browse/CRUNCH-571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14956912#comment-14956912
]
Tom White commented on CRUNCH-571:
----------------------------------
To reproduce run
{code}
import org.apache.crunch.scrunch.Mem
case class A(x: Int)
val pColOfA = Mem.collectionOf(A(1), A(2))
pColOfA.filter(row => row.x == 1)
{code}
which results in
{noformat}
java.lang.IllegalStateException: anon$2 named 'null' cannot be serialized
at
org.apache.crunch.impl.mem.collect.MemCollection.verifySerializable(MemCollection.java:115)
at
org.apache.crunch.impl.mem.collect.MemCollection.parallelDo(MemCollection.java:177)
at
org.apache.crunch.impl.mem.collect.MemCollection.parallelDo(MemCollection.java:171)
at
org.apache.crunch.impl.mem.collect.MemCollection.parallelDo(MemCollection.java:166)
at
org.apache.crunch.impl.mem.collect.MemCollection.filter(MemCollection.java:323)
at
org.apache.crunch.scrunch.PCollectionLike$class.filter(PCollectionLike.scala:106)
at org.apache.crunch.scrunch.PCollection.filter(PCollection.scala:29)
... 33 elided
Caused by: org.apache.commons.lang.SerializationException:
java.lang.ClassNotFoundException: $anonfun$1
at
org.apache.crunch.impl.mem.collect.MemCollection.deserialize(MemCollection.java:142)
at
org.apache.crunch.impl.mem.collect.MemCollection.deserialize(MemCollection.java:161)
at
org.apache.crunch.impl.mem.collect.MemCollection.verifySerializable(MemCollection.java:113)
... 39 more
Caused by: java.lang.ClassNotFoundException: $anonfun$1
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:274)
at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:625)
at
org.apache.crunch.impl.mem.collect.MemCollection$1.resolveClass(MemCollection.java:135)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1612)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
at
org.apache.crunch.impl.mem.collect.MemCollection.deserialize(MemCollection.java:139)
... 41 more
{noformat}
> Scrunch functions fail serialization check in the REPL
> ------------------------------------------------------
>
> Key: CRUNCH-571
> URL: https://issues.apache.org/jira/browse/CRUNCH-571
> Project: Crunch
> Issue Type: Bug
> Components: Scrunch
> Affects Versions: 0.12.0
> Reporter: Tom White
> Assignee: Tom White
>
> Calling {{pcol.filter(row => row.x == 1)}} results in
> {{java.lang.IllegalStateException: anon$2 named 'null' cannot be serialized}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
