[
https://issues.apache.org/jira/browse/BEAM-5646?focusedWorklogId=157119&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-157119
]
ASF GitHub Bot logged work on BEAM-5646:
----------------------------------------
Author: ASF GitHub Bot
Created on: 22/Oct/18 19:44
Start Date: 22/Oct/18 19:44
Worklog Time Spent: 10m
Work Description: amaliujia commented on a change in pull request #6765:
[BEAM-5646] Fix quality and hashcode for bytes in Row.
URL: https://github.com/apache/beam/pull/6765#discussion_r227055056
##########
File path: sdks/java/core/src/main/java/org/apache/beam/sdk/values/Row.java
##########
@@ -346,13 +347,48 @@ public boolean equals(Object o) {
return false;
}
Row other = (Row) o;
- return Objects.equals(getSchema(), other.getSchema())
- && Objects.equals(getValues(), other.getValues());
+
+ if (!Objects.equals(getSchema(), other.getSchema())) {
+ return false;
+ }
+
+ Iterator e1 = getValues().iterator();
+ Iterator e2 = other.getValues().iterator();
+ while (e1.hasNext() && e2.hasNext()) {
+ Object o1 = e1.next();
+ Object o2 = e2.next();
+
+ if (o1 instanceof byte[]) {
+ if (!Arrays.equals((byte[]) o1, (byte[]) o2)) {
+ return false;
+ }
+ } else if (!(o1 == null ? o2 == null : o1.equals(o2))) {
+ return false;
+ }
+ }
+
+ return !(e1.hasNext() || e2.hasNext());
}
@Override
public int hashCode() {
Review comment:
Updated
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 157119)
Time Spent: 1h 50m (was: 1h 40m)
> Equality is broken for Rows with BYTES field
> --------------------------------------------
>
> Key: BEAM-5646
> URL: https://issues.apache.org/jira/browse/BEAM-5646
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql
> Affects Versions: 2.7.0
> Reporter: Gleb Kanterov
> Assignee: Xu Mingmin
> Priority: Major
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> The problem is with `org.apache.beam.sdk.values.Row#equals` and `hashCode`.
> Java arrays do reference equality instead of comparing contents. Row stores
> fields of type BYTES as byte[].
> These failing tests illustrate the problem:
> {code:java}
> @Test
> public void testByteArrayEquality() {
> byte[] a0 = new byte[16];
> byte[] b0 = new byte[16];
> Schema schema = Schema.of(Schema.Field.of("bytes", Schema.FieldType.BYTES));
> Row a = Row.withSchema(schema).addValue(a0).build();
> Row b = Row.withSchema(schema).addValue(b0).build();
> Assert.assertEquals(a, b);
> }
> @Test
> public void testByteBufferEquality() {
> byte[] a0 = new byte[16];
> byte[] b0 = new byte[16];
> Schema schema = Schema.of(Schema.Field.of("bytes", Schema.FieldType.BYTES));
> Row a = Row.withSchema(schema).addValue(ByteBuffer.wrap(a0)).build();
> Row b = Row.withSchema(schema).addValue(ByteBuffer.wrap(b0)).build();
> Assert.assertEquals(a, b);
> }
> {code}
>
> Option 1. Fix by storing `byte[]` as `ByteBuffer`, or something more simple
> that doesn't have offsets. `Row#getValue` will return this type, and for
> consistency, it would be preferable to change `Row#getBytes` in an
> incompatible way to be consistent with `Row#getValue` because that's how it
> behaves for the rest of the methods.
>
> Option 2. Do the same as Spark does, add `if (x instanceof byte[])` to
> `equals`. The problem in Spark is that `hashCode` implementation isn't
> consistent with `equals`, see SPARK-25122.
>
> Option 3. Consider it as intended behavior, and fix
> `RowCoder#consistentWithEquals` implementation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
