Re: Bug broadcasting objects (serialization issue)

Hi, I get a new similar bug when broadcasting a list of integers if this list is made unmodifiable, elements = Collections.unmodifiableList(elements); I include this code to reproduce the result, public class WordCountExample { public static void main(String[] args) throws

Re: Bug broadcasting objects (serialization issue)

Thanks for clarifying the "eager serialization". By serializing and deserializing explicitly (eagerly) we can raise better Exceptions to notify the user of non-serializable classes. > BTW: There is an opportunity to fix two problems with one patch: The > framesize overflow for the input format,

Bug broadcasting objects (serialization issue)

Hi, I get a bug when trying to broadcast a list of integers created with the primitive "Arrays.asList(...)". For example, if you try to run this "wordcount" example, you can reproduce the bug. public class WordCountExample { public static void main(String[] args) throws Exception {

Re: Bug broadcasting objects (serialization issue)

Hi Andreas, Thank you for reporting the problem and including the code to reproduce the problem. I think there is a problem with the class serialization or deserialization. Arrays.asList uses a private ArrayList class (java.util.Arrays$ArrayList) which is not the one you would normally use

Re: Bug broadcasting objects (serialization issue)

Yes, even serialize in the constructor. Then the failure (if serialization does not work) comes immediately. On Wed, Sep 2, 2015 at 4:02 PM, Maximilian Michels wrote: > Nice suggestion. So you want to serialize and deserialize the InputFormats > on the Client to check whether

Re: Bug broadcasting objects (serialization issue)

Nice suggestion. So you want to serialize and deserialize the InputFormats on the Client to check whether they can be transferred correctly? Merely serializing is not enough because the above Exception occurs during deserialization. On Wed, Sep 2, 2015 at 2:29 PM, Stephan Ewen

Re: Bug broadcasting objects (serialization issue)

Here's the JIRA issue: https://issues.apache.org/jira/browse/FLINK-2608 On Wed, Sep 2, 2015 at 12:49 PM, Maximilian Michels wrote: > Hi Andreas, > > Thank you for reporting the problem and including the code to reproduce > the problem. I think there is a problem with the class

Re: Bug broadcasting objects (serialization issue)

We should try to improve the exception here. More people will run into this issue and the exception should help them understand it well. How about we do eager serialization into a set of byte arrays? Then the serializability issue comes immediately when the program is constructed, rather than

Re: Bug broadcasting objects (serialization issue)

I see. Manual serialization implies also manual deserialization (on the workers only), which would give a better exception. BTW: There is an opportunity to fix two problems with one patch: The framesize overflow for the input format, and the serialization. On Wed, Sep 2, 2015 at 4:16 PM,