[
https://issues.apache.org/jira/browse/CRUNCH-571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tom White updated CRUNCH-571:
-----------------------------
Attachment: CRUNCH-571.patch
The problem is that {{ObjectInputStream}} (which is used by
{{SerializationUtils}} from Apache Commons) does some magic tricks to find a
classloader, but these don't work in the REPL context. The problem is explained
well in https://issues.scala-lang.org/browse/SI-2403, however the fix there
doesn't quite work in our case, since the Scrunch classes are loaded by the
application class loader, not the REPL classloader, so the class for the
anonymous function can still not be found.
The REPL does set the thread context classloader, however, so a slight
adaptation of the approach in SI-2403 does work if we try the thread context
classloader, which is what this patch does.
There's no unit test as this problem only manifests itself in the REPL, however
I have manually tested the fix with success.
> Scrunch functions fail serialization check in the REPL
> ------------------------------------------------------
>
> Key: CRUNCH-571
> URL: https://issues.apache.org/jira/browse/CRUNCH-571
> Project: Crunch
> Issue Type: Bug
> Components: Scrunch
> Affects Versions: 0.12.0
> Reporter: Tom White
> Assignee: Tom White
> Attachments: CRUNCH-571.patch
>
>
> Calling {{pcol.filter(row => row.x == 1)}} results in
> {{java.lang.IllegalStateException: anon$2 named 'null' cannot be serialized}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
