Re: Design rational behind copying via serializing in flink runner

2020-09-07 Thread Maximilian Michels
Hey Teodor, Copying is the default behavior. This is tunable via the pipeline option 'objectReuse', i.e. 'objectReuse=true'. The option is disabled by default because users may not be aware of object reuse and recycle objects in their process functions which will have unexpected side

Design rational behind copying via serializing in flink runner

2020-08-28 Thread Teodor Spæren
Hey! First time posting to a mailing list, hope I did it correctly :) I'm writing a master thesis at the University of Oslo and right now I'm looking at the performance overhead of using Beam with the Flink runnner versus plain Flink. I've written a simple program, a custom source outputing