[jira] [Commented] (AVRO-2058) ReflectData#isNonStringMap returns true for Utf8 keys
[
https://issues.apache.org/jira/browse/AVRO-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16715598#comment-16715598
]
ASF GitHub Bot commented on AVRO-2058:
--
samschlegel commented on issue #237: [AVRO-2058] ReflectData#isNonStringMap
returns true for Utf8 keys
URL: https://github.com/apache/avro/pull/237#issuecomment-445983786
Ah, thanks @dkulp! Sorry this got away from me :sweat_smile:
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> ReflectData#isNonStringMap returns true for Utf8 keys
> -
>
> Key: AVRO-2058
> URL: https://issues.apache.org/jira/browse/AVRO-2058
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.8.2
>Reporter: Sam Schlegel
>Assignee: Sam Schlegel
>Priority: Critical
> Fix For: 1.9.0
>
> Attachments: AVRO-2058.patch
>
>
> Since {{Utf8}} does not have an {{Stringable}} notation, and is not in
> {{SpecificData#stringableClasses}}, {{ReflectData#isNonStringMap}} returns
> true. This also causes {{ReflectData#isArray}} to return true for maps with
> Utf8 keys, and thus {{GenericData#resolveUnion}} fails as well. This
> ultimately causes {{ReflectData#write}} to fail for schemas that contain a
> union that contains a map, where the data uses Utf8 for strings.
> This following test case reproduces the issue:
> {code:java}
> @Test public void testUnionWithMapWithUtf8Keys() {
> Schema s = new Schema.Parser().parse
> ("[\"null\", {\"type\":\"map\",\"values\":\"float\"}]");
> GenericData data = ReflectData.get();
> HashMap map = new HashMap();
> map.put(new Utf8("foo"), 1.0f);
> assertEquals(1, data.resolveUnion(s, map));
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AVRO-2058) ReflectData#isNonStringMap returns true for Utf8 keys
[
https://issues.apache.org/jira/browse/AVRO-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16715500#comment-16715500
]
ASF subversion and git services commented on AVRO-2058:
---
Commit 269d0ff7d3025cf3370023e353a373916dd67433 in avro's branch
refs/heads/master from [~dkulp]
[ https://gitbox.apache.org/repos/asf?p=avro.git;h=269d0ff ]
[AVRO-2058] Fix issues with Utf8 keys maps being treated as non-strings
This closes #237
commit 8852da40e7a0ee58d9b027173a4972e7e71b432d
Author: Sam Schlegel
Date: Wed Jul 26 19:20:51 2017 -0700
Add failing test
> ReflectData#isNonStringMap returns true for Utf8 keys
> -
>
> Key: AVRO-2058
> URL: https://issues.apache.org/jira/browse/AVRO-2058
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.8.2
>Reporter: Sam Schlegel
>Assignee: Sam Schlegel
>Priority: Critical
> Attachments: AVRO-2058.patch
>
>
> Since {{Utf8}} does not have an {{Stringable}} notation, and is not in
> {{SpecificData#stringableClasses}}, {{ReflectData#isNonStringMap}} returns
> true. This also causes {{ReflectData#isArray}} to return true for maps with
> Utf8 keys, and thus {{GenericData#resolveUnion}} fails as well. This
> ultimately causes {{ReflectData#write}} to fail for schemas that contain a
> union that contains a map, where the data uses Utf8 for strings.
> This following test case reproduces the issue:
> {code:java}
> @Test public void testUnionWithMapWithUtf8Keys() {
> Schema s = new Schema.Parser().parse
> ("[\"null\", {\"type\":\"map\",\"values\":\"float\"}]");
> GenericData data = ReflectData.get();
> HashMap map = new HashMap();
> map.put(new Utf8("foo"), 1.0f);
> assertEquals(1, data.resolveUnion(s, map));
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AVRO-2058) ReflectData#isNonStringMap returns true for Utf8 keys
[
https://issues.apache.org/jira/browse/AVRO-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16715501#comment-16715501
]
ASF GitHub Bot commented on AVRO-2058:
--
dkulp closed pull request #237: [AVRO-2058] ReflectData#isNonStringMap returns
true for Utf8 keys
URL: https://github.com/apache/avro/pull/237
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/lang/java/avro/src/main/java/org/apache/avro/util/Utf8.java
b/lang/java/avro/src/main/java/org/apache/avro/util/Utf8.java
index 27d11a316..8f8369ddb 100644
--- a/lang/java/avro/src/main/java/org/apache/avro/util/Utf8.java
+++ b/lang/java/avro/src/main/java/org/apache/avro/util/Utf8.java
@@ -21,10 +21,12 @@
import java.io.UnsupportedEncodingException;
import org.apache.avro.io.BinaryData;
+import org.apache.avro.reflect.Stringable;
/** A Utf8 string. Unlike {@link String}, instances are mutable. This is more
* efficient than {@link String} when reading or writing a sequence of values,
* as a single instance may be reused. */
+@Stringable
public class Utf8 implements Comparable, CharSequence {
private static final byte[] EMPTY = new byte[0];
private static final Charset UTF8 = Charset.forName("UTF-8");
diff --git
a/lang/java/avro/src/test/java/org/apache/avro/reflect/TestReflect.java
b/lang/java/avro/src/test/java/org/apache/avro/reflect/TestReflect.java
index afe7f08ae..35b957a70 100644
--- a/lang/java/avro/src/test/java/org/apache/avro/reflect/TestReflect.java
+++ b/lang/java/avro/src/test/java/org/apache/avro/reflect/TestReflect.java
@@ -47,6 +47,7 @@
import org.apache.avro.io.Encoder;
import org.apache.avro.io.EncoderFactory;
import org.apache.avro.reflect.TestReflect.SampleRecord.AnotherSampleRecord;
+import org.apache.avro.util.Utf8;
import org.codehaus.jackson.node.NullNode;
import org.junit.Test;
@@ -123,6 +124,15 @@
assertEquals(1, data.resolveUnion(s, new HashMap()));
}
+ @Test public void testUnionWithMapWithUtf8Keys() {
+Schema s = new Schema.Parser().parse
+ ("[\"null\", {\"type\":\"map\",\"values\":\"float\"}]");
+GenericData data = ReflectData.get();
+HashMap map = new HashMap();
+map.put(new Utf8("foo"), 1.0f);
+assertEquals(1, data.resolveUnion(s, map));
+ }
+
@Test public void testUnionWithFixed() {
Schema s = new Schema.Parser().parse
("[\"null\", {\"type\":\"fixed\",\"name\":\"f\",\"size\":1}]");
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> ReflectData#isNonStringMap returns true for Utf8 keys
> -
>
> Key: AVRO-2058
> URL: https://issues.apache.org/jira/browse/AVRO-2058
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.8.2
>Reporter: Sam Schlegel
>Assignee: Sam Schlegel
>Priority: Critical
> Attachments: AVRO-2058.patch
>
>
> Since {{Utf8}} does not have an {{Stringable}} notation, and is not in
> {{SpecificData#stringableClasses}}, {{ReflectData#isNonStringMap}} returns
> true. This also causes {{ReflectData#isArray}} to return true for maps with
> Utf8 keys, and thus {{GenericData#resolveUnion}} fails as well. This
> ultimately causes {{ReflectData#write}} to fail for schemas that contain a
> union that contains a map, where the data uses Utf8 for strings.
> This following test case reproduces the issue:
> {code:java}
> @Test public void testUnionWithMapWithUtf8Keys() {
> Schema s = new Schema.Parser().parse
> ("[\"null\", {\"type\":\"map\",\"values\":\"float\"}]");
> GenericData data = ReflectData.get();
> HashMap map = new HashMap();
> map.put(new Utf8("foo"), 1.0f);
> assertEquals(1, data.resolveUnion(s, map));
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AVRO-2058) ReflectData#isNonStringMap returns true for Utf8 keys
[
https://issues.apache.org/jira/browse/AVRO-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524852#comment-16524852
]
ASF GitHub Bot commented on AVRO-2058:
--
Weait commented on issue #237: [AVRO-2058] ReflectData#isNonStringMap returns
true for Utf8 keys
URL: https://github.com/apache/avro/pull/237#issuecomment-400614936
@samschlegel, what's the state of updating the PR with @nandorKollar's
comments? Need someone to take over this pull request or can you finish within
a reasonable amount of time?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> ReflectData#isNonStringMap returns true for Utf8 keys
> -
>
> Key: AVRO-2058
> URL: https://issues.apache.org/jira/browse/AVRO-2058
> Project: Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.8.2
>Reporter: Sam Schlegel
>Assignee: Sam Schlegel
>Priority: Critical
> Attachments: AVRO-2058.patch
>
>
> Since {{Utf8}} does not have an {{Stringable}} notation, and is not in
> {{SpecificData#stringableClasses}}, {{ReflectData#isNonStringMap}} returns
> true. This also causes {{ReflectData#isArray}} to return true for maps with
> Utf8 keys, and thus {{GenericData#resolveUnion}} fails as well. This
> ultimately causes {{ReflectData#write}} to fail for schemas that contain a
> union that contains a map, where the data uses Utf8 for strings.
> This following test case reproduces the issue:
> {code:java}
> @Test public void testUnionWithMapWithUtf8Keys() {
> Schema s = new Schema.Parser().parse
> ("[\"null\", {\"type\":\"map\",\"values\":\"float\"}]");
> GenericData data = ReflectData.get();
> HashMap map = new HashMap();
> map.put(new Utf8("foo"), 1.0f);
> assertEquals(1, data.resolveUnion(s, map));
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AVRO-2058) ReflectData#isNonStringMap returns true for Utf8 keys
[
https://issues.apache.org/jira/browse/AVRO-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16224871#comment-16224871
]
Nandor Kollar commented on AVRO-2058:
-
[~SamSchlegel] I commented your PR, could you please have a look at my comment?
> ReflectData#isNonStringMap returns true for Utf8 keys
> -
>
> Key: AVRO-2058
> URL: https://issues.apache.org/jira/browse/AVRO-2058
> Project: Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.8.2
>Reporter: Sam Schlegel
>Assignee: Sam Schlegel
>Priority: Critical
> Attachments: AVRO-2058.patch
>
>
> Since {{Utf8}} does not have an {{Stringable}} notation, and is not in
> {{SpecificData#stringableClasses}}, {{ReflectData#isNonStringMap}} returns
> true. This also causes {{ReflectData#isArray}} to return true for maps with
> Utf8 keys, and thus {{GenericData#resolveUnion}} fails as well. This
> ultimately causes {{ReflectData#write}} to fail for schemas that contain a
> union that contains a map, where the data uses Utf8 for strings.
> This following test case reproduces the issue:
> {code:java}
> @Test public void testUnionWithMapWithUtf8Keys() {
> Schema s = new Schema.Parser().parse
> ("[\"null\", {\"type\":\"map\",\"values\":\"float\"}]");
> GenericData data = ReflectData.get();
> HashMap map = new HashMap();
> map.put(new Utf8("foo"), 1.0f);
> assertEquals(1, data.resolveUnion(s, map));
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AVRO-2058) ReflectData#isNonStringMap returns true for Utf8 keys
[
https://issues.apache.org/jira/browse/AVRO-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16217691#comment-16217691
]
Sam Schlegel commented on AVRO-2058:
What needs to be done to move this along?
> ReflectData#isNonStringMap returns true for Utf8 keys
> -
>
> Key: AVRO-2058
> URL: https://issues.apache.org/jira/browse/AVRO-2058
> Project: Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.8.2
>Reporter: Sam Schlegel
>Assignee: Sam Schlegel
>Priority: Critical
> Attachments: AVRO-2058.patch
>
>
> Since {{Utf8}} does not have an {{Stringable}} notation, and is not in
> {{SpecificData#stringableClasses}}, {{ReflectData#isNonStringMap}} returns
> true. This also causes {{ReflectData#isArray}} to return true for maps with
> Utf8 keys, and thus {{GenericData#resolveUnion}} fails as well. This
> ultimately causes {{ReflectData#write}} to fail for schemas that contain a
> union that contains a map, where the data uses Utf8 for strings.
> This following test case reproduces the issue:
> {code:java}
> @Test public void testUnionWithMapWithUtf8Keys() {
> Schema s = new Schema.Parser().parse
> ("[\"null\", {\"type\":\"map\",\"values\":\"float\"}]");
> GenericData data = ReflectData.get();
> HashMap map = new HashMap();
> map.put(new Utf8("foo"), 1.0f);
> assertEquals(1, data.resolveUnion(s, map));
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AVRO-2058) ReflectData#isNonStringMap returns true for Utf8 keys
[
https://issues.apache.org/jira/browse/AVRO-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16103636#comment-16103636
]
ASF GitHub Bot commented on AVRO-2058:
--
GitHub user samschlegel opened a pull request:
https://github.com/apache/avro/pull/237
[AVRO-2058] ReflectData#isNonStringMap returns true for Utf8 keys
https://issues.apache.org/jira/browse/AVRO-2058
Since `Utf8` does not have an `Stringable` notation, and is not in
`SpecificData#stringableClasses`, `ReflectData#isNonStringMap` returns true.
This also causes `ReflectData#isArray` to return true for maps with Utf8 keys,
and thus `GenericData#resolveUnion` fails as well. This ultimately causes
`ReflectData#write` to fail for schemas that contain a union that contains a
map, where the data uses Utf8 for strings.
This following test case reproduces the issue:
```java
@Test public void testUnionWithMapWithUtf8Keys() {
Schema s = new Schema.Parser().parse
("[\"null\", {\"type\":\"map\",\"values\":\"float\"}]");
GenericData data = ReflectData.get();
HashMap map = new HashMap();
map.put(new Utf8("foo"), 1.0f);
assertEquals(1, data.resolveUnion(s, map));
}
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/samschlegel/avro master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/avro/pull/237.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #237
commit 8852da40e7a0ee58d9b027173a4972e7e71b432d
Author: Sam Schlegel
Date: 2017-07-27T02:20:51Z
Add failing test
commit 15fbb40944dc15a37569aa6a3d2e93db0e085af0
Author: Sam Schlegel
Date: 2017-07-27T02:23:21Z
Add Stringable annotation
> ReflectData#isNonStringMap returns true for Utf8 keys
> -
>
> Key: AVRO-2058
> URL: https://issues.apache.org/jira/browse/AVRO-2058
> Project: Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.8.2
>Reporter: Sam Schlegel
>Priority: Critical
> Attachments: AVRO-2058.patch
>
>
> Since {{Utf8}} does not have an {{Stringable}} notation, and is not in
> {{SpecificData#stringableClasses}}, {{ReflectData#isNonStringMap}} returns
> true. This also causes {{ReflectData#isArray}} to return true for maps with
> Utf8 keys, and thus {{GenericData#resolveUnion}} fails as well. This
> ultimately causes {{ReflectData#write}} to fail for schemas that contain a
> union that contains a map, where the data uses Utf8 for strings.
> This following test case reproduces the issue:
> {code:java}
> @Test public void testUnionWithMapWithUtf8Keys() {
> Schema s = new Schema.Parser().parse
> ("[\"null\", {\"type\":\"map\",\"values\":\"float\"}]");
> GenericData data = ReflectData.get();
> HashMap map = new HashMap();
> map.put(new Utf8("foo"), 1.0f);
> assertEquals(1, data.resolveUnion(s, map));
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
