[jira] [Commented] (BEAM-8364) SchemaCoder inconsistent equality behavior for POJO
[
https://issues.apache.org/jira/browse/BEAM-8364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17548324#comment-17548324
]
Danny McCormick commented on BEAM-8364:
---
This issue has been migrated to https://github.com/apache/beam/issues/19966
> SchemaCoder inconsistent equality behavior for POJO
> ---
>
> Key: BEAM-8364
> URL: https://issues.apache.org/jira/browse/BEAM-8364
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql, sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Neville Li
>Priority: P3
> Labels: Clarified
> Time Spent: 1h
> Remaining Estimate: 0h
>
> One can create a {{SchemaCoder}} for arbitrary type {{T}} with
> {{SchemaCoder.of(schema, toRowFunction, fromRowFunction)}}. However, in cases
> where {{T}} lacks proper equality behavior, i.e. POJO, the result coder still
> returns true for {{consistentWithEquals}} and {{structuralValue}}s that fail
> equality check.
> This test reproduces the issue.
> {code:java}
> import org.apache.beam.sdk.schemas.Schema;
> import org.apache.beam.sdk.schemas.SchemaCoder;
> import org.apache.beam.sdk.values.Row;
> import org.junit.Test;
> import org.junit.runner.RunWith;
> import org.junit.runners.JUnit4;
> import java.nio.charset.Charset;
> import static org.junit.Assert.*;
> @RunWith(JUnit4.class)
> public class SchemaCoderTest {
> public static class Pojo {
> private final byte[] bytes;
> private final String id;
> public Pojo(byte[] bytes, String id) {
> this.bytes = bytes;
> this.id = id;
> }
> public byte[] getBytes() {
> return bytes;
> }
> public String getId() {
> return id;
> }
> }
> @Test
> public void testCoder() {
> Schema schema =
> Schema.builder().addByteArrayField("bytes").addStringField("id").build();
> SchemaCoder coder = SchemaCoder.of(
> schema,
> t -> Row.withSchema(schema).addValues(t.getBytes(),
> t.getId()).build(),
> r -> new Pojo(r.getBytes("bytes"), r.getString("id")));
> Pojo p1 = new Pojo("hello".getBytes(Charset.forName("UTF-8")), "world");
> Pojo p2 = new Pojo("hello".getBytes(Charset.forName("UTF-8")), "world");
> assertNotEquals(p1, p2); // EXPECTED, p1.equals(p2) == false
> assertFalse(coder.consistentWithEquals()); // FAIL, returns true
> assertEquals(coder.structuralValue(p1), coder.structuralValue(p2)); //
> FAIL
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (BEAM-8364) SchemaCoder inconsistent equality behavior for POJO
[
https://issues.apache.org/jira/browse/BEAM-8364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17185984#comment-17185984
]
Beam JIRA Bot commented on BEAM-8364:
-
This issue was marked "stale-assigned" and has not received a public comment in
7 days. It is now automatically unassigned. If you are still working on it, you
can assign it to yourself again. Please also give an update about the status of
the work.
> SchemaCoder inconsistent equality behavior for POJO
> ---
>
> Key: BEAM-8364
> URL: https://issues.apache.org/jira/browse/BEAM-8364
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql, sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Neville Li
>Priority: P3
> Time Spent: 40m
> Remaining Estimate: 0h
>
> One can create a {{SchemaCoder}} for arbitrary type {{T}} with
> {{SchemaCoder.of(schema, toRowFunction, fromRowFunction)}}. However, in cases
> where {{T}} lacks proper equality behavior, i.e. POJO, the result coder still
> returns true for {{consistentWithEquals}} and {{structuralValue}}s that fail
> equality check.
> This test reproduces the issue.
> {code:java}
> import org.apache.beam.sdk.schemas.Schema;
> import org.apache.beam.sdk.schemas.SchemaCoder;
> import org.apache.beam.sdk.values.Row;
> import org.junit.Test;
> import org.junit.runner.RunWith;
> import org.junit.runners.JUnit4;
> import java.nio.charset.Charset;
> import static org.junit.Assert.*;
> @RunWith(JUnit4.class)
> public class SchemaCoderTest {
> public static class Pojo {
> private final byte[] bytes;
> private final String id;
> public Pojo(byte[] bytes, String id) {
> this.bytes = bytes;
> this.id = id;
> }
> public byte[] getBytes() {
> return bytes;
> }
> public String getId() {
> return id;
> }
> }
> @Test
> public void testCoder() {
> Schema schema =
> Schema.builder().addByteArrayField("bytes").addStringField("id").build();
> SchemaCoder coder = SchemaCoder.of(
> schema,
> t -> Row.withSchema(schema).addValues(t.getBytes(),
> t.getId()).build(),
> r -> new Pojo(r.getBytes("bytes"), r.getString("id")));
> Pojo p1 = new Pojo("hello".getBytes(Charset.forName("UTF-8")), "world");
> Pojo p2 = new Pojo("hello".getBytes(Charset.forName("UTF-8")), "world");
> assertNotEquals(p1, p2); // EXPECTED, p1.equals(p2) == false
> assertFalse(coder.consistentWithEquals()); // FAIL, returns true
> assertEquals(coder.structuralValue(p1), coder.structuralValue(p2)); //
> FAIL
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-8364) SchemaCoder inconsistent equality behavior for POJO
[
https://issues.apache.org/jira/browse/BEAM-8364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17181330#comment-17181330
]
Beam JIRA Bot commented on BEAM-8364:
-
This issue is assigned but has not received an update in 30 days so it has been
labeled "stale-assigned". If you are still working on the issue, please give an
update and remove the label. If you are no longer working on the issue, please
unassign so someone else may work on it. In 7 days the issue will be
automatically unassigned.
> SchemaCoder inconsistent equality behavior for POJO
> ---
>
> Key: BEAM-8364
> URL: https://issues.apache.org/jira/browse/BEAM-8364
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql, sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Neville Li
>Assignee: Brian Hulette
>Priority: P3
> Labels: stale-assigned
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> One can create a {{SchemaCoder}} for arbitrary type {{T}} with
> {{SchemaCoder.of(schema, toRowFunction, fromRowFunction)}}. However, in cases
> where {{T}} lacks proper equality behavior, i.e. POJO, the result coder still
> returns true for {{consistentWithEquals}} and {{structuralValue}}s that fail
> equality check.
> This test reproduces the issue.
> {code:java}
> import org.apache.beam.sdk.schemas.Schema;
> import org.apache.beam.sdk.schemas.SchemaCoder;
> import org.apache.beam.sdk.values.Row;
> import org.junit.Test;
> import org.junit.runner.RunWith;
> import org.junit.runners.JUnit4;
> import java.nio.charset.Charset;
> import static org.junit.Assert.*;
> @RunWith(JUnit4.class)
> public class SchemaCoderTest {
> public static class Pojo {
> private final byte[] bytes;
> private final String id;
> public Pojo(byte[] bytes, String id) {
> this.bytes = bytes;
> this.id = id;
> }
> public byte[] getBytes() {
> return bytes;
> }
> public String getId() {
> return id;
> }
> }
> @Test
> public void testCoder() {
> Schema schema =
> Schema.builder().addByteArrayField("bytes").addStringField("id").build();
> SchemaCoder coder = SchemaCoder.of(
> schema,
> t -> Row.withSchema(schema).addValues(t.getBytes(),
> t.getId()).build(),
> r -> new Pojo(r.getBytes("bytes"), r.getString("id")));
> Pojo p1 = new Pojo("hello".getBytes(Charset.forName("UTF-8")), "world");
> Pojo p2 = new Pojo("hello".getBytes(Charset.forName("UTF-8")), "world");
> assertNotEquals(p1, p2); // EXPECTED, p1.equals(p2) == false
> assertFalse(coder.consistentWithEquals()); // FAIL, returns true
> assertEquals(coder.structuralValue(p1), coder.structuralValue(p2)); //
> FAIL
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-8364) SchemaCoder inconsistent equality behavior for POJO
[
https://issues.apache.org/jira/browse/BEAM-8364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17161487#comment-17161487
]
Brian Hulette commented on BEAM-8364:
-
I still want to look at this. There's a PR up now but there's an open question
about it: https://github.com/apache/beam/pull/11981#discussion_r442966866.
[~reuvenlax] would prefer we just require types that use SchemaCoder to have an
equals method defined. I like that, but it would be nice if we could enforce it
so that users don't just get undefined behavior when there is no equals method.
It's possible to check that a class has equals defined using reflection, would
it be enough to just throw an error when a type doesn't have equals defined?
What if the type has a bad equals method?
> SchemaCoder inconsistent equality behavior for POJO
> ---
>
> Key: BEAM-8364
> URL: https://issues.apache.org/jira/browse/BEAM-8364
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql, sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Neville Li
>Assignee: Brian Hulette
>Priority: P3
> Labels: beam-fixit, stale-assigned
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> One can create a {{SchemaCoder}} for arbitrary type {{T}} with
> {{SchemaCoder.of(schema, toRowFunction, fromRowFunction)}}. However, in cases
> where {{T}} lacks proper equality behavior, i.e. POJO, the result coder still
> returns true for {{consistentWithEquals}} and {{structuralValue}}s that fail
> equality check.
> This test reproduces the issue.
> {code:java}
> import org.apache.beam.sdk.schemas.Schema;
> import org.apache.beam.sdk.schemas.SchemaCoder;
> import org.apache.beam.sdk.values.Row;
> import org.junit.Test;
> import org.junit.runner.RunWith;
> import org.junit.runners.JUnit4;
> import java.nio.charset.Charset;
> import static org.junit.Assert.*;
> @RunWith(JUnit4.class)
> public class SchemaCoderTest {
> public static class Pojo {
> private final byte[] bytes;
> private final String id;
> public Pojo(byte[] bytes, String id) {
> this.bytes = bytes;
> this.id = id;
> }
> public byte[] getBytes() {
> return bytes;
> }
> public String getId() {
> return id;
> }
> }
> @Test
> public void testCoder() {
> Schema schema =
> Schema.builder().addByteArrayField("bytes").addStringField("id").build();
> SchemaCoder coder = SchemaCoder.of(
> schema,
> t -> Row.withSchema(schema).addValues(t.getBytes(),
> t.getId()).build(),
> r -> new Pojo(r.getBytes("bytes"), r.getString("id")));
> Pojo p1 = new Pojo("hello".getBytes(Charset.forName("UTF-8")), "world");
> Pojo p2 = new Pojo("hello".getBytes(Charset.forName("UTF-8")), "world");
> assertNotEquals(p1, p2); // EXPECTED, p1.equals(p2) == false
> assertFalse(coder.consistentWithEquals()); // FAIL, returns true
> assertEquals(coder.structuralValue(p1), coder.structuralValue(p2)); //
> FAIL
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-8364) SchemaCoder inconsistent equality behavior for POJO
[
https://issues.apache.org/jira/browse/BEAM-8364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17161397#comment-17161397
]
Beam JIRA Bot commented on BEAM-8364:
-
This issue is assigned but has not received an update in 30 days so it has been
labeled "stale-assigned". If you are still working on the issue, please give an
update and remove the label. If you are no longer working on the issue, please
unassign so someone else may work on it. In 7 days the issue will be
automatically unassigned.
> SchemaCoder inconsistent equality behavior for POJO
> ---
>
> Key: BEAM-8364
> URL: https://issues.apache.org/jira/browse/BEAM-8364
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql, sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Neville Li
>Assignee: Brian Hulette
>Priority: P3
> Labels: beam-fixit, stale-assigned
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> One can create a {{SchemaCoder}} for arbitrary type {{T}} with
> {{SchemaCoder.of(schema, toRowFunction, fromRowFunction)}}. However, in cases
> where {{T}} lacks proper equality behavior, i.e. POJO, the result coder still
> returns true for {{consistentWithEquals}} and {{structuralValue}}s that fail
> equality check.
> This test reproduces the issue.
> {code:java}
> import org.apache.beam.sdk.schemas.Schema;
> import org.apache.beam.sdk.schemas.SchemaCoder;
> import org.apache.beam.sdk.values.Row;
> import org.junit.Test;
> import org.junit.runner.RunWith;
> import org.junit.runners.JUnit4;
> import java.nio.charset.Charset;
> import static org.junit.Assert.*;
> @RunWith(JUnit4.class)
> public class SchemaCoderTest {
> public static class Pojo {
> private final byte[] bytes;
> private final String id;
> public Pojo(byte[] bytes, String id) {
> this.bytes = bytes;
> this.id = id;
> }
> public byte[] getBytes() {
> return bytes;
> }
> public String getId() {
> return id;
> }
> }
> @Test
> public void testCoder() {
> Schema schema =
> Schema.builder().addByteArrayField("bytes").addStringField("id").build();
> SchemaCoder coder = SchemaCoder.of(
> schema,
> t -> Row.withSchema(schema).addValues(t.getBytes(),
> t.getId()).build(),
> r -> new Pojo(r.getBytes("bytes"), r.getString("id")));
> Pojo p1 = new Pojo("hello".getBytes(Charset.forName("UTF-8")), "world");
> Pojo p2 = new Pojo("hello".getBytes(Charset.forName("UTF-8")), "world");
> assertNotEquals(p1, p2); // EXPECTED, p1.equals(p2) == false
> assertFalse(coder.consistentWithEquals()); // FAIL, returns true
> assertEquals(coder.structuralValue(p1), coder.structuralValue(p2)); //
> FAIL
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-8364) SchemaCoder inconsistent equality behavior for POJO
[
https://issues.apache.org/jira/browse/BEAM-8364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17131207#comment-17131207
]
Beam JIRA Bot commented on BEAM-8364:
-
This issue was marked "stale-assigned" and has not received a public comment in
7 days. It is now automatically unassigned. If you are still working on it, you
can assign it to yourself again. Please also give an update about the status of
the work.
> SchemaCoder inconsistent equality behavior for POJO
> ---
>
> Key: BEAM-8364
> URL: https://issues.apache.org/jira/browse/BEAM-8364
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql, sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Neville Li
>Priority: P3
>
> One can create a {{SchemaCoder}} for arbitrary type {{T}} with
> {{SchemaCoder.of(schema, toRowFunction, fromRowFunction)}}. However, in cases
> where {{T}} lacks proper equality behavior, i.e. POJO, the result coder still
> returns true for {{consistentWithEquals}} and {{structuralValue}}s that fail
> equality check.
> This test reproduces the issue.
> {code:java}
> import org.apache.beam.sdk.schemas.Schema;
> import org.apache.beam.sdk.schemas.SchemaCoder;
> import org.apache.beam.sdk.values.Row;
> import org.junit.Test;
> import org.junit.runner.RunWith;
> import org.junit.runners.JUnit4;
> import java.nio.charset.Charset;
> import static org.junit.Assert.*;
> @RunWith(JUnit4.class)
> public class SchemaCoderTest {
> public static class Pojo {
> private final byte[] bytes;
> private final String id;
> public Pojo(byte[] bytes, String id) {
> this.bytes = bytes;
> this.id = id;
> }
> public byte[] getBytes() {
> return bytes;
> }
> public String getId() {
> return id;
> }
> }
> @Test
> public void testCoder() {
> Schema schema =
> Schema.builder().addByteArrayField("bytes").addStringField("id").build();
> SchemaCoder coder = SchemaCoder.of(
> schema,
> t -> Row.withSchema(schema).addValues(t.getBytes(),
> t.getId()).build(),
> r -> new Pojo(r.getBytes("bytes"), r.getString("id")));
> Pojo p1 = new Pojo("hello".getBytes(Charset.forName("UTF-8")), "world");
> Pojo p2 = new Pojo("hello".getBytes(Charset.forName("UTF-8")), "world");
> assertNotEquals(p1, p2); // EXPECTED, p1.equals(p2) == false
> assertFalse(coder.consistentWithEquals()); // FAIL, returns true
> assertEquals(coder.structuralValue(p1), coder.structuralValue(p2)); //
> FAIL
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-8364) SchemaCoder inconsistent equality behavior for POJO
[
https://issues.apache.org/jira/browse/BEAM-8364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121858#comment-17121858
]
Kenneth Knowles commented on BEAM-8364:
---
This issue is assigned but has not received an update in 30 days so it has been
labeled "stale-assigned". If you are still working on the issue, please give an
update and remove the label. If you are no longer working on the issue, please
unassign so someone else may work on it. In 7 days the issue will be
automatically unassigned.
> SchemaCoder inconsistent equality behavior for POJO
> ---
>
> Key: BEAM-8364
> URL: https://issues.apache.org/jira/browse/BEAM-8364
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql, sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Neville Li
>Assignee: Brian Hulette
>Priority: P3
> Labels: stale-assigned
>
> One can create a {{SchemaCoder}} for arbitrary type {{T}} with
> {{SchemaCoder.of(schema, toRowFunction, fromRowFunction)}}. However, in cases
> where {{T}} lacks proper equality behavior, i.e. POJO, the result coder still
> returns true for {{consistentWithEquals}} and {{structuralValue}}s that fail
> equality check.
> This test reproduces the issue.
> {code:java}
> import org.apache.beam.sdk.schemas.Schema;
> import org.apache.beam.sdk.schemas.SchemaCoder;
> import org.apache.beam.sdk.values.Row;
> import org.junit.Test;
> import org.junit.runner.RunWith;
> import org.junit.runners.JUnit4;
> import java.nio.charset.Charset;
> import static org.junit.Assert.*;
> @RunWith(JUnit4.class)
> public class SchemaCoderTest {
> public static class Pojo {
> private final byte[] bytes;
> private final String id;
> public Pojo(byte[] bytes, String id) {
> this.bytes = bytes;
> this.id = id;
> }
> public byte[] getBytes() {
> return bytes;
> }
> public String getId() {
> return id;
> }
> }
> @Test
> public void testCoder() {
> Schema schema =
> Schema.builder().addByteArrayField("bytes").addStringField("id").build();
> SchemaCoder coder = SchemaCoder.of(
> schema,
> t -> Row.withSchema(schema).addValues(t.getBytes(),
> t.getId()).build(),
> r -> new Pojo(r.getBytes("bytes"), r.getString("id")));
> Pojo p1 = new Pojo("hello".getBytes(Charset.forName("UTF-8")), "world");
> Pojo p2 = new Pojo("hello".getBytes(Charset.forName("UTF-8")), "world");
> assertNotEquals(p1, p2); // EXPECTED, p1.equals(p2) == false
> assertFalse(coder.consistentWithEquals()); // FAIL, returns true
> assertEquals(coder.structuralValue(p1), coder.structuralValue(p2)); //
> FAIL
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-8364) SchemaCoder inconsistent equality behavior for POJO
[
https://issues.apache.org/jira/browse/BEAM-8364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16948851#comment-16948851
]
Neville Li commented on BEAM-8364:
--
Yeah I agree it's hard to guarantee that {{consistentWithEquals}} reports
correctly. Always return {{false}} seems reasonable. What about making
{{structuralValue}} always return {{Row}} also, converting with
{{toRowFunction(T)}} if necessary?
> SchemaCoder inconsistent equality behavior for POJO
> ---
>
> Key: BEAM-8364
> URL: https://issues.apache.org/jira/browse/BEAM-8364
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql, sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Neville Li
>Assignee: Brian Hulette
>Priority: Minor
>
> One can create a {{SchemaCoder}} for arbitrary type {{T}} with
> {{SchemaCoder.of(schema, toRowFunction, fromRowFunction)}}. However, in cases
> where {{T}} lacks proper equality behavior, i.e. POJO, the result coder still
> returns true for {{consistentWithEquals}} and {{structuralValue}}s that fail
> equality check.
> This test reproduces the issue.
> {code:java}
> import org.apache.beam.sdk.schemas.Schema;
> import org.apache.beam.sdk.schemas.SchemaCoder;
> import org.apache.beam.sdk.values.Row;
> import org.junit.Test;
> import org.junit.runner.RunWith;
> import org.junit.runners.JUnit4;
> import java.nio.charset.Charset;
> import static org.junit.Assert.*;
> @RunWith(JUnit4.class)
> public class SchemaCoderTest {
> public static class Pojo {
> private final byte[] bytes;
> private final String id;
> public Pojo(byte[] bytes, String id) {
> this.bytes = bytes;
> this.id = id;
> }
> public byte[] getBytes() {
> return bytes;
> }
> public String getId() {
> return id;
> }
> }
> @Test
> public void testCoder() {
> Schema schema =
> Schema.builder().addByteArrayField("bytes").addStringField("id").build();
> SchemaCoder coder = SchemaCoder.of(
> schema,
> t -> Row.withSchema(schema).addValues(t.getBytes(),
> t.getId()).build(),
> r -> new Pojo(r.getBytes("bytes"), r.getString("id")));
> Pojo p1 = new Pojo("hello".getBytes(Charset.forName("UTF-8")), "world");
> Pojo p2 = new Pojo("hello".getBytes(Charset.forName("UTF-8")), "world");
> assertNotEquals(p1, p2); // EXPECTED, p1.equals(p2) == false
> assertFalse(coder.consistentWithEquals()); // FAIL, returns true
> assertEquals(coder.structuralValue(p1), coder.structuralValue(p2)); //
> FAIL
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-8364) SchemaCoder inconsistent equality behavior for POJO
[
https://issues.apache.org/jira/browse/BEAM-8364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16948826#comment-16948826
]
Brian Hulette commented on BEAM-8364:
-
I'm not sure what the appropriate fix is here. Should we always return
{{false}} from {{consistentWithEquals}}, unless we can be 100% sure that the
encoded type has a good equals that we can be consistent with (e.g.
{{SchemaCoder}} generated with {{AutoValueSchema}})?
That seems like a reasonable approach.. but I'm not familiar enough with
consistentWithEquals/structuralValue to know what sort of impact that would
have.
> SchemaCoder inconsistent equality behavior for POJO
> ---
>
> Key: BEAM-8364
> URL: https://issues.apache.org/jira/browse/BEAM-8364
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql, sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Neville Li
>Assignee: Brian Hulette
>Priority: Minor
>
> One can create a {{SchemaCoder}} for arbitrary type {{T}} with
> {{SchemaCoder.of(schema, toRowFunction, fromRowFunction)}}. However, in cases
> where {{T}} lacks proper equality behavior, i.e. POJO, the result coder still
> returns true for {{consistentWithEquals}} and {{structuralValue}}s that fail
> equality check.
> This test reproduces the issue.
> {code:java}
> import org.apache.beam.sdk.schemas.Schema;
> import org.apache.beam.sdk.schemas.SchemaCoder;
> import org.apache.beam.sdk.values.Row;
> import org.junit.Test;
> import org.junit.runner.RunWith;
> import org.junit.runners.JUnit4;
> import java.nio.charset.Charset;
> import static org.junit.Assert.*;
> @RunWith(JUnit4.class)
> public class SchemaCoderTest {
> public static class Pojo {
> private final byte[] bytes;
> private final String id;
> public Pojo(byte[] bytes, String id) {
> this.bytes = bytes;
> this.id = id;
> }
> public byte[] getBytes() {
> return bytes;
> }
> public String getId() {
> return id;
> }
> }
> @Test
> public void testCoder() {
> Schema schema =
> Schema.builder().addByteArrayField("bytes").addStringField("id").build();
> SchemaCoder coder = SchemaCoder.of(
> schema,
> t -> Row.withSchema(schema).addValues(t.getBytes(),
> t.getId()).build(),
> r -> new Pojo(r.getBytes("bytes"), r.getString("id")));
> Pojo p1 = new Pojo("hello".getBytes(Charset.forName("UTF-8")), "world");
> Pojo p2 = new Pojo("hello".getBytes(Charset.forName("UTF-8")), "world");
> assertNotEquals(p1, p2); // EXPECTED, p1.equals(p2) == false
> assertFalse(coder.consistentWithEquals()); // FAIL, returns true
> assertEquals(coder.structuralValue(p1), coder.structuralValue(p2)); //
> FAIL
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
