AW: Guarantees for object reuse modes and documentation

Hi Ken, yes, that’s exactly how it work in Flink as well. The object reuse move dues not affect how chaining is done. Best, Fabian Von: Ken Krugler Gesendet: Freitag, 4. März 2016 01:54 An: dev@flink.apache.org Betreff: RE: Guarantees for object reuse modes and documentation Hi Gábor, When

RE: Guarantees for object reuse modes and documentation

is done, which prevents a problem with output tuple re-use. Is this the same model used by Flink? Thanks for clarifying, -- Ken > From: Gábor Gévay > Sent: February 20, 2016 4:04:09am PST > To: dev@flink.apache.org > Subject: Re: Guarantees for object reuse modes and documentation > &

Re: Guarantees for object reuse modes and documentation

ould be cached. If there's a >>> non-primitive object then changes to this in the upstream operation >>> obviously bork the cached data. >>> > >>> > Net-net is that it we wanted a way to find out, from inside an >>> operation, whether we needed to make a

Re: Guarantees for object reuse modes and documentation

d a way to find out, from inside an >> operation, whether we needed to make a deep copy of the output Tuple. But >> that doesn't exist (yet), so we have some utility code to check if a deep >> copy is needed (non-primitive types), and if so then it auto-clones the >> T

Re: Guarantees for object reuse modes and documentation

nes the > Tuple. Which isn't very efficient, but for most of our workflows we only > have primitive types. > > > > -- Ken > > > >> From: Fabian Hueske > >> Sent: February 17, 2016 9:17:27am PST > >> To: dev@flink.apache.org > >> Subject: Guara

Re: Guarantees for object reuse modes and documentation

efficient, but for most of our workflows we only have primitive types. > > -- Ken > >> From: Fabian Hueske >> Sent: February 17, 2016 9:17:27am PST >> To: dev@flink.apache.org >> Subject: Guarantees for object reuse modes and documentation >> >> Hi, >&

RE: Guarantees for object reuse modes and documentation

am PST > To: dev@flink.apache.org > Subject: Guarantees for object reuse modes and documentation > > Hi, > > > > Flink's DataSet API features a configuration parameter called > enableObjectReuse(). If activated, Flink's runtime will create fewer > objects which results in b

Re: Guarantees for object reuse modes and documentation

Hi Fabian, I would only add to your citations Stephan's comment [1] concerning the design, implementation, and use of object reuse. I see two separate concerns addressed in code. First, as Stephan noted, for certain classes deserialization is sufficiently expensive relative to object creation

Re: Guarantees for object reuse modes and documentation

Judging from our chaining condition ds.getPushChainDriverClass() != null && !(pred instanceof NAryUnionPlanNode) &&// first op after union is stand-alone, because union is merged !(pred instanceof BulkPartialSolutionPlanNode) &&// partial solution merges anyways !(pred instanceof

Re: Guarantees for object reuse modes and documentation

Thanks Matthias. Maybe I should clarify, that I do not want to change the guarantees for the enableObjectReuse mode, but for the disableObjectReuse mode. The rules for the enableObjectReuse mode should remain the same. 2016-02-18 9:37 GMT+01:00 Matthias J. Sax : > Hi, > > I

Guarantees for object reuse modes and documentation

Hi, Flink's DataSet API features a configuration parameter called enableObjectReuse(). If activated, Flink's runtime will create fewer objects which results in better performance and lower garbage collection overhead. Depending on whether the configuration switch is enabled or not, user