Re: Does a PCollection element class override equals() ?

[Kenneth Knowles]

Last I checked, equals() was used only as a shortcut. If the two are not
equals() then their encoded forms should be checked. If neither equals()
nor the coder can work for this, you will have a bad time.

On Sun, Jun 18, 2017 at 10:14 PM, Jean-Baptiste Onofré 
wrote:

> Hi team,
>
> The direct runner checks that there's no mutation on elements in a
> PCollection thanks to ImmutabilityEnforcementFactory.
> This factory uses CodedValueMutationDetector to detect if an element has
> been changed or not.
>
> The CodedValueMutationDetector uses equals (in the
> verifyUnmodifiedThrowingCheckedExceptions() method) for that.
>
> However, in an IO on which I'm working on, the element class doesn't
> override equals and it fails with:
>
> org.apache.beam.sdk.util.IllegalMutationException: PTransform
> AmqpIO.Write/ParDo(Write)/ParMultiDo(Write) illegaly mutated value
> Message{body=AmqpValue{Test 0}} of class class
> org.apache.qpid.proton.message.impl.MessageImpl. Input values must not be
> mutated in any way.
>
> So, basically my question is:
>
> 1. Do I need to wrap the message in a custom wrapper overriding the
> equals() method ?
> 2. Maybe we could improve a bit the checker in the direct runner ?
>
> Thanks
> Regards
> JB
> --
> Jean-Baptiste Onofré
> jbono...@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>

Does a PCollection element class override equals() ?

[Jean-Baptiste Onofré]

Hi team,

The direct runner checks that there's no mutation on elements in a PCollection 
thanks to ImmutabilityEnforcementFactory.
This factory uses CodedValueMutationDetector to detect if an element has been 
changed or not.


The CodedValueMutationDetector uses equals (in the 
verifyUnmodifiedThrowingCheckedExceptions() method) for that.


However, in an IO on which I'm working on, the element class doesn't override 
equals and it fails with:


org.apache.beam.sdk.util.IllegalMutationException: PTransform 
AmqpIO.Write/ParDo(Write)/ParMultiDo(Write) illegaly mutated value 
Message{body=AmqpValue{Test 0}} of class class 
org.apache.qpid.proton.message.impl.MessageImpl. Input values must not be 
mutated in any way.


So, basically my question is:

1. Do I need to wrap the message in a custom wrapper overriding the equals() 
method ?

2. Maybe we could improve a bit the checker in the direct runner ?

Thanks
Regards
JB
--
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Jenkins build is back to normal : beam_Release_NightlySnapshot #451

[Apache Jenkins Server]

See