[jira] [Commented] (AVRO-2775) JacksonUtils: exception when calling toJsonNode()
[
https://issues.apache.org/jira/browse/AVRO-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259306#comment-17259306
]
Andy Le commented on AVRO-2775:
---
[~rskraba] I think allowing the behaviour when generating defaults is enough.
But would you elaborate more on
{quote}For my point #2 above (with Age going in, but Map coming out), maybe
it's enough to just have this clearly documented on the JsonProperties
interface.
{quote}
> JacksonUtils: exception when calling toJsonNode()
> --
>
> Key: AVRO-2775
> URL: https://issues.apache.org/jira/browse/AVRO-2775
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.2
>Reporter: Andy Le
>Assignee: Andy Le
>Priority: Major
>
> I've got a simple test as followed
> {code:java}
> public class TestJacksonUtils {
> public static class Age{
> public int value = 9;
> }
> @Test
> public void testToJson(){
> Map kv = new HashMap<>();
> kv.put("age", 9);
> JsonNode node1 = JacksonUtils.toJsonNode(kv); // -> This is OK
> Object obj = new Age();
> JsonNode node2 = JacksonUtils.toJsonNode(obj); // -> This will trigger an
> exception
> }
> }
> {code}
> When I ran the test:
> {noformat}
> org.apache.avro.AvroRuntimeException: Unknown datum class: class
> org.apache.avro.util.internal.TestJacksonUtils$Age
> at
> org.apache.avro.util.internal.JacksonUtils.toJson(JacksonUtils.java:87)
> at
> org.apache.avro.util.internal.JacksonUtils.toJsonNode(JacksonUtils.java:48)
> at
> org.apache.avro.util.internal.TestJacksonUtils.testToJson(TestJacksonUtils.java:20)
> {noformat}
> I've read the code & tests for JacksonUtils. Instead of raising exceptions at
> [line
> #87|https://github.com/apache/avro/blob/master/lang/java/avro/src/main/java/org/apache/avro/util/internal/JacksonUtils.java#L87],
> I see we can auto convert objects into maps, every thing's gonna fine.
> My question is:
> - Is raising exception acceptable?
> - Any other way to have `toJsonNode` for general objects?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (AVRO-2775) JacksonUtils: exception when calling toJsonNode()
[
https://issues.apache.org/jira/browse/AVRO-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Le reassigned AVRO-2775:
-
Assignee: Andy Le
> JacksonUtils: exception when calling toJsonNode()
> --
>
> Key: AVRO-2775
> URL: https://issues.apache.org/jira/browse/AVRO-2775
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.2
>Reporter: Andy Le
>Assignee: Andy Le
>Priority: Major
>
> I've got a simple test as followed
> {code:java}
> public class TestJacksonUtils {
> public static class Age{
> public int value = 9;
> }
> @Test
> public void testToJson(){
> Map kv = new HashMap<>();
> kv.put("age", 9);
> JsonNode node1 = JacksonUtils.toJsonNode(kv); // -> This is OK
> Object obj = new Age();
> JsonNode node2 = JacksonUtils.toJsonNode(obj); // -> This will trigger an
> exception
> }
> }
> {code}
> When I ran the test:
> {noformat}
> org.apache.avro.AvroRuntimeException: Unknown datum class: class
> org.apache.avro.util.internal.TestJacksonUtils$Age
> at
> org.apache.avro.util.internal.JacksonUtils.toJson(JacksonUtils.java:87)
> at
> org.apache.avro.util.internal.JacksonUtils.toJsonNode(JacksonUtils.java:48)
> at
> org.apache.avro.util.internal.TestJacksonUtils.testToJson(TestJacksonUtils.java:20)
> {noformat}
> I've read the code & tests for JacksonUtils. Instead of raising exceptions at
> [line
> #87|https://github.com/apache/avro/blob/master/lang/java/avro/src/main/java/org/apache/avro/util/internal/JacksonUtils.java#L87],
> I see we can auto convert objects into maps, every thing's gonna fine.
> My question is:
> - Is raising exception acceptable?
> - Any other way to have `toJsonNode` for general objects?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2775) JacksonUtils: exception when calling toJsonNode()
[
https://issues.apache.org/jira/browse/AVRO-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258991#comment-17258991
]
Andy Le commented on AVRO-2775:
---
[~rskraba] My use case is: I want to extract default values for fields. Please
see my story here: [https://github.com/anhldbk/avro-fork]
For the time being, the code below throws exceptions:
{code:java}
import org.apache.avro.Schema;
import org.apache.avro.reflect.ReflectData;
public class AvroPlay {
public static class Age{
public int value = 9;
}
public static class Person{
public String name = "Andy";
public Age age = new Age();
}
public static void main(String[] args) {
ReflectData reflector = new ReflectData().setDefaultsGenerated(true);
Schema schema = reflector.getSchema(Person.class);
System.out.println(schema.toString(true));
}
}
{code}
When I run it, I've got an exception of:
{noformat}
Exception in thread "main" org.apache.avro.AvroRuntimeException: Unknown datum
class: class org.example.AvroPlay$Age
at
org.apache.avro.util.internal.JacksonUtils.toJson(JacksonUtils.java:96)
at
org.apache.avro.util.internal.JacksonUtils.toJsonNode(JacksonUtils.java:53)
at org.apache.avro.Schema$Field.(Schema.java:581)
at
org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:747)
at
org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:328)
at
org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:325)
at java.lang.ClassValue.getFromHashMap(ClassValue.java:227)
at java.lang.ClassValue.getFromBackup(ClassValue.java:209)
at java.lang.ClassValue.get(ClassValue.java:115)
at
org.apache.avro.specific.SpecificData.getSchema(SpecificData.java:339)
at org.example.AvroPlay.main(AvroPlay.java:18)
{noformat}
So is it reasonable to modify the behavior of
`JacksonUtils.toJson(JacksonUtils.java:96)` a little bit?
> JacksonUtils: exception when calling toJsonNode()
> --
>
> Key: AVRO-2775
> URL: https://issues.apache.org/jira/browse/AVRO-2775
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.2
>Reporter: Andy Le
>Priority: Major
>
> I've got a simple test as followed
> {code:java}
> public class TestJacksonUtils {
> public static class Age{
> public int value = 9;
> }
> @Test
> public void testToJson(){
> Map kv = new HashMap<>();
> kv.put("age", 9);
> JsonNode node1 = JacksonUtils.toJsonNode(kv); // -> This is OK
> Object obj = new Age();
> JsonNode node2 = JacksonUtils.toJsonNode(obj); // -> This will trigger an
> exception
> }
> }
> {code}
> When I ran the test:
> {noformat}
> org.apache.avro.AvroRuntimeException: Unknown datum class: class
> org.apache.avro.util.internal.TestJacksonUtils$Age
> at
> org.apache.avro.util.internal.JacksonUtils.toJson(JacksonUtils.java:87)
> at
> org.apache.avro.util.internal.JacksonUtils.toJsonNode(JacksonUtils.java:48)
> at
> org.apache.avro.util.internal.TestJacksonUtils.testToJson(TestJacksonUtils.java:20)
> {noformat}
> I've read the code & tests for JacksonUtils. Instead of raising exceptions at
> [line
> #87|https://github.com/apache/avro/blob/master/lang/java/avro/src/main/java/org/apache/avro/util/internal/JacksonUtils.java#L87],
> I see we can auto convert objects into maps, every thing's gonna fine.
> My question is:
> - Is raising exception acceptable?
> - Any other way to have `toJsonNode` for general objects?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2847) Travis Windows/C# build is sometimes flaky
[
https://issues.apache.org/jira/browse/AVRO-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135692#comment-17135692
]
Andy Le commented on AVRO-2847:
---
[~rskraba] Do you think we have problem with Travis? Their supports for Windows
are experimental
> Travis Windows/C# build is sometimes flaky
> --
>
> Key: AVRO-2847
> URL: https://issues.apache.org/jira/browse/AVRO-2847
> Project: Apache Avro
> Issue Type: Task
>Reporter: Ryan Skraba
>Priority: Major
> Attachments: travis-windows-failure-example-log.txt
>
>
> We use the [https://travis-ci.org/github/apache/avro/] build to validate PRs,
> but the Windows build occasionally fails with an error message:
> {code}
> Build FAILED.
> C:\Users\travis\build\apache\avro\lang\csharp\src\apache\test\Avro.test.csproj
> : error NU1603: Apache.Avro depends on Newtonsoft.Json (>= 10.0.3) but
> Newtonsoft.Json 10.0.3 was not found. An approximate best match of
> Newtonsoft.Json 11.0.2 was resolved.
> [C:\Users\travis\build\apache\avro\lang\csharp\Avro.sln]
> C:\Users\travis\build\apache\avro\lang\csharp\src\apache\test\Avro.test.csproj
> : error NU1101: Unable to find package nunit. No packages exist with this id
> in source(s): Microsoft Visual Studio Offline Packages
> [C:\Users\travis\build\apache\avro\lang\csharp\Avro.sln]
> C:\Users\travis\build\apache\avro\lang\csharp\src\apache\test\Avro.test.csproj
> : error NU1101: Unable to find package nunit3testadapter. No packages exist
> with this id in source(s): Microsoft Visual Studio Offline Packages
> [C:\Users\travis\build\apache\avro\lang\csharp\Avro.sln]
> C:\Users\travis\build\apache\avro\lang\csharp\src\apache\test\Avro.test.csproj
> : error NU1101: Unable to find package NUnit.ConsoleRunner. No packages
> exist with this id in source(s): Microsoft Visual Studio Offline Packages
> [C:\Users\travis\build\apache\avro\lang\csharp\Avro.sln]
> C:\Users\travis\build\apache\avro\lang\csharp\src\apache\test\Avro.test.csproj
> : error NU1101: Unable to find package Microsoft.NET.Test.Sdk. No packages
> exist with this id in source(s): Microsoft Visual Studio Offline Packages
> [C:\Users\travis\build\apache\avro\lang\csharp\Avro.sln]
> C:\Users\travis\build\apache\avro\lang\csharp\src\apache\test\Avro.test.csproj
> : error NU1101: Unable to find package System.CodeDom. No packages exist
> with this id in source(s): Microsoft Visual Studio Offline Packages
> [C:\Users\travis\build\apache\avro\lang\csharp\Avro.sln]
> 0 Warning(s)
> {code}
> Relaunching the job usually succeeds.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-1329) Get per-symbol doc for enums
[
https://issues.apache.org/jira/browse/AVRO-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17103636#comment-17103636
]
Andy Le commented on AVRO-1329:
---
[~unchuckable] looks great. I think we should have changes in our spec first
> Get per-symbol doc for enums
>
>
> Key: AVRO-1329
> URL: https://issues.apache.org/jira/browse/AVRO-1329
> Project: Apache Avro
> Issue Type: Improvement
> Components: doc
>Affects Versions: 1.7.4
>Reporter: Felix GV
>Priority: Minor
>
> It would be nice to have the ability to add documentation for each symbol of
> an enum.
> Doug Cutting, quoted from the mailing list:
> Documentation per enum symbol is not currently supported, but would not be
> difficult to add. Please file an issue in Jira if you'd like to see this. For
> compatibility, in Json, this would probably appear as a parallel array of
> documentation strings, e.g., something like:
> ("name": "Foo", "type":"enum", "doc":"an enum", "symbols":["X","Y"],
> "symbols-doc":["X is X", "Y is Y"]}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (AVRO-2808) Java: ReflectData incorrectly handles hidden fields
[
https://issues.apache.org/jira/browse/AVRO-2808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Le updated AVRO-2808:
--
Issue Type: Improvement (was: Bug)
> Java: ReflectData incorrectly handles hidden fields
> ---
>
> Key: AVRO-2808
> URL: https://issues.apache.org/jira/browse/AVRO-2808
> Project: Apache Avro
> Issue Type: Improvement
> Components: java
>Affects Versions: 1.9.2
>Reporter: Andy Le
>Assignee: Andy Le
>Priority: Major
>
> Hi guys,
> I've got the following test:
> {code:java}
> public ReflectTest{
> public class Definition {
> public Map tokens;
> }
> public enum Type {
> A,
> B,
> C
> }
> @Test
> public void testAvro2805() {
> Schema schema = ReflectData.get().getSchema(Definition.class);
> final String schemaString = schema.toString(true);
> System.out.println(schemaString);
> }
> }
> {code}
> When I ran the test, an exception is raised:
> {noformat}
> org.apache.avro.SchemaParseException: Illegal character in: this$0
> at org.apache.avro.Schema.validateName(Schema.java:1530)
> at org.apache.avro.Schema.access$400(Schema.java:87)
> at org.apache.avro.Schema$Field.(Schema.java:518)
> at org.apache.avro.Schema$Field.(Schema.java:557)
> at
> org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:683)
> at
> org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:335)
> at
> org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:332)
> {noformat}
> From the log, you may see that: we should not handle Java hidden field
> `this$0` which is used to hold reference to outer class instances.
> This issue is somehow related to [this
> one|https://issues.apache.org/jira/browse/AVRO-2805]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work started] (AVRO-2808) Java: ReflectData incorrectly handles hidden fields
[
https://issues.apache.org/jira/browse/AVRO-2808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on AVRO-2808 started by Andy Le.
-
> Java: ReflectData incorrectly handles hidden fields
> ---
>
> Key: AVRO-2808
> URL: https://issues.apache.org/jira/browse/AVRO-2808
> Project: Apache Avro
> Issue Type: Improvement
> Components: java
>Affects Versions: 1.9.2
>Reporter: Andy Le
>Assignee: Andy Le
>Priority: Major
>
> Hi guys,
> I've got the following test:
> {code:java}
> public ReflectTest{
> public class Definition {
> public Map tokens;
> }
> public enum Type {
> A,
> B,
> C
> }
> @Test
> public void testAvro2805() {
> Schema schema = ReflectData.get().getSchema(Definition.class);
> final String schemaString = schema.toString(true);
> System.out.println(schemaString);
> }
> }
> {code}
> When I ran the test, an exception is raised:
> {noformat}
> org.apache.avro.SchemaParseException: Illegal character in: this$0
> at org.apache.avro.Schema.validateName(Schema.java:1530)
> at org.apache.avro.Schema.access$400(Schema.java:87)
> at org.apache.avro.Schema$Field.(Schema.java:518)
> at org.apache.avro.Schema$Field.(Schema.java:557)
> at
> org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:683)
> at
> org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:335)
> at
> org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:332)
> {noformat}
> From the log, you may see that: we should not handle Java hidden field
> `this$0` which is used to hold reference to outer class instances.
> This issue is somehow related to [this
> one|https://issues.apache.org/jira/browse/AVRO-2805]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (AVRO-2808) Java: ReflectData incorrectly handles hidden fields
[
https://issues.apache.org/jira/browse/AVRO-2808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Le reassigned AVRO-2808:
-
Assignee: Andy Le
> Java: ReflectData incorrectly handles hidden fields
> ---
>
> Key: AVRO-2808
> URL: https://issues.apache.org/jira/browse/AVRO-2808
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.2
>Reporter: Andy Le
>Assignee: Andy Le
>Priority: Major
>
> Hi guys,
> I've got the following test:
> {code:java}
> public ReflectTest{
> public class Definition {
> public Map tokens;
> }
> public enum Type {
> A,
> B,
> C
> }
> @Test
> public void testAvro2805() {
> Schema schema = ReflectData.get().getSchema(Definition.class);
> final String schemaString = schema.toString(true);
> System.out.println(schemaString);
> }
> }
> {code}
> When I ran the test, an exception is raised:
> {noformat}
> org.apache.avro.SchemaParseException: Illegal character in: this$0
> at org.apache.avro.Schema.validateName(Schema.java:1530)
> at org.apache.avro.Schema.access$400(Schema.java:87)
> at org.apache.avro.Schema$Field.(Schema.java:518)
> at org.apache.avro.Schema$Field.(Schema.java:557)
> at
> org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:683)
> at
> org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:335)
> at
> org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:332)
> {noformat}
> From the log, you may see that: we should not handle Java hidden field
> `this$0` which is used to hold reference to outer class instances.
> This issue is somehow related to [this
> one|https://issues.apache.org/jira/browse/AVRO-2805]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2278) GenericData.Record field getter not correct
[
https://issues.apache.org/jira/browse/AVRO-2278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17103007#comment-17103007
]
Andy Le commented on AVRO-2278:
---
[~zolyfarkas] [~rskraba] I think there are 2 other options:
* Opt #1. Introduce Schema.exist(key) to verify if a key is valid, no change
to Schema.get(key)
* Opt #2. Introduce a more consistent way in Schema.fetch(key) and mark
Schema.get(key) as deprecated.
> GenericData.Record field getter not correct
> ---
>
> Key: AVRO-2278
> URL: https://issues.apache.org/jira/browse/AVRO-2278
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.8.2, 1.9.2
>Reporter: Zoltan Farkas
>Assignee: Zoltan Farkas
>Priority: Major
>
> Currently the get field implementation is not correct in GenericData.Record:
> at:
> https://github.com/apache/avro/blob/master/lang/java/avro/src/main/java/org/apache/avro/generic/GenericData.java#L209
> {code}
>@Override public Object get(String key) {
> Field field = schema.getField(key);
> if (field == null) return null;
> return values[field.pos()];
> }
> {code}
> The method returns null when a field is not present, making it impossible to
> distinguish between:
> field value = null
> and
> field does not exist.
> A more "correct" implementation would be:
> {code}
> @Override public Object get(String key) {
> Field field = schema.getField(key);
> if (field == null) {
> throw new IllegalArgumentException("Invalid field " + key);
> }
> return values[field.pos()];
> }
> {code}
> this will make the behavior consistent with put which will throw a exception
> when setting a non existent field.
> when I make this change in my fork, some bugs in unit tests showed up
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2808) Java: ReflectData incorrectly handles hidden fields
[
https://issues.apache.org/jira/browse/AVRO-2808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17100882#comment-17100882
]
Andy Le commented on AVRO-2808:
---
[~rskraba] nice to see you here.
> I think this should be the expected behaviour for non-static inner classes!
Fine for me. The question is: should we customize the exception to display a
more meaningful message to users?
> Java: ReflectData incorrectly handles hidden fields
> ---
>
> Key: AVRO-2808
> URL: https://issues.apache.org/jira/browse/AVRO-2808
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.2
>Reporter: Andy Le
>Priority: Major
>
> Hi guys,
> I've got the following test:
> {code:java}
> public ReflectTest{
> public class Definition {
> public Map tokens;
> }
> public enum Type {
> A,
> B,
> C
> }
> @Test
> public void testAvro2805() {
> Schema schema = ReflectData.get().getSchema(Definition.class);
> final String schemaString = schema.toString(true);
> System.out.println(schemaString);
> }
> }
> {code}
> When I ran the test, an exception is raised:
> {noformat}
> org.apache.avro.SchemaParseException: Illegal character in: this$0
> at org.apache.avro.Schema.validateName(Schema.java:1530)
> at org.apache.avro.Schema.access$400(Schema.java:87)
> at org.apache.avro.Schema$Field.(Schema.java:518)
> at org.apache.avro.Schema$Field.(Schema.java:557)
> at
> org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:683)
> at
> org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:335)
> at
> org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:332)
> {noformat}
> From the log, you may see that: we should not handle Java hidden field
> `this$0` which is used to hold reference to outer class instances.
> This issue is somehow related to [this
> one|https://issues.apache.org/jira/browse/AVRO-2805]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2778) Java: Reflect raises exceptions while dealing with non-static classes
[
https://issues.apache.org/jira/browse/AVRO-2778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17098199#comment-17098199
]
Andy Le commented on AVRO-2778:
---
This issue is related to https://issues.apache.org/jira/browse/AVRO-2808. So I
close this one.
> Java: Reflect raises exceptions while dealing with non-static classes
> -
>
> Key: AVRO-2778
> URL: https://issues.apache.org/jira/browse/AVRO-2778
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.2
>Reporter: Andy Le
>Priority: Major
>
> Hi guys,
> I've got the following simple code:
> {code:java}
> public class TestReflect {
> public class Human{
> String name = "Andy";
> }
> @Test
> public void testNonStaticClasses(){
> ReflectData.get().getSchema(Human.class) ;
> }
> }
> {code}
> When I run the test, an exception is spawn:
> {noformat}
> org.apache.avro.SchemaParseException: Illegal character in: this$0
> at org.apache.avro.Schema.validateName(Schema.java:1530)
> at org.apache.avro.Schema.access$400(Schema.java:87)
> at org.apache.avro.Schema$Field.(Schema.java:518)
> at org.apache.avro.Schema$Field.(Schema.java:557)
> at
> org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:645)
> at
> org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:335)
> at
> org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:1)
> at java.lang.ClassValue.getFromHashMap(ClassValue.java:227)
> at java.lang.ClassValue.getFromBackup(ClassValue.java:209)
> at java.lang.ClassValue.get(ClassValue.java:115)
> at
> org.apache.avro.specific.SpecificData.getSchema(SpecificData.java:346)
> ...
> {noformat}
> It seems to me that: Avro Reflect does NOT effectively handle non-static
> classes.
> Should I make a PR against this issue?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (AVRO-2778) Java: Reflect raises exceptions while dealing with non-static classes
[
https://issues.apache.org/jira/browse/AVRO-2778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Le resolved AVRO-2778.
---
Assignee: Andy Le
Resolution: Duplicate
> Java: Reflect raises exceptions while dealing with non-static classes
> -
>
> Key: AVRO-2778
> URL: https://issues.apache.org/jira/browse/AVRO-2778
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.2
>Reporter: Andy Le
>Assignee: Andy Le
>Priority: Major
>
> Hi guys,
> I've got the following simple code:
> {code:java}
> public class TestReflect {
> public class Human{
> String name = "Andy";
> }
> @Test
> public void testNonStaticClasses(){
> ReflectData.get().getSchema(Human.class) ;
> }
> }
> {code}
> When I run the test, an exception is spawn:
> {noformat}
> org.apache.avro.SchemaParseException: Illegal character in: this$0
> at org.apache.avro.Schema.validateName(Schema.java:1530)
> at org.apache.avro.Schema.access$400(Schema.java:87)
> at org.apache.avro.Schema$Field.(Schema.java:518)
> at org.apache.avro.Schema$Field.(Schema.java:557)
> at
> org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:645)
> at
> org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:335)
> at
> org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:1)
> at java.lang.ClassValue.getFromHashMap(ClassValue.java:227)
> at java.lang.ClassValue.getFromBackup(ClassValue.java:209)
> at java.lang.ClassValue.get(ClassValue.java:115)
> at
> org.apache.avro.specific.SpecificData.getSchema(SpecificData.java:346)
> ...
> {noformat}
> It seems to me that: Avro Reflect does NOT effectively handle non-static
> classes.
> Should I make a PR against this issue?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2723) Avro Java: Obtaining default field values for POJO objects with ReflectData
[
https://issues.apache.org/jira/browse/AVRO-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095484#comment-17095484
]
Andy Le commented on AVRO-2723:
---
[~rskraba] [~iemejia] pls do you have time to review my PR? Master Cutting
seems to be busy.
> Avro Java: Obtaining default field values for POJO objects with ReflectData
> ---
>
> Key: AVRO-2723
> URL: https://issues.apache.org/jira/browse/AVRO-2723
> Project: Apache Avro
> Issue Type: New Feature
> Components: java
>Affects Versions: 1.9.1
>Reporter: Andy Le
>Priority: Critical
> Attachments: Screen Shot 2020-03-08 at 16.13.29.png
>
>
> Hi guys,
>
> I've got a simple app using Avro Reflection:
>
> {code:java}
> public class App {
> public static void main(String[] args) {
> testReflection();
> }
> static class User {
> public String first = "Andy";
> public String last = "Le";
> }
> static void testReflection(){
> // get the reflected schema for packets
> Schema schema = ReflectData.AllowNull.get().getSchema(User.class);
> System.out.println(schema.toString(true));
> }
> {code}
> The output on console will be:
> {noformat}
> {
> "type" : "record",
> "name" : "User",
> "namespace" : "App",
> "fields" : [ {
> "name" : "first",
> "type" : [ "null", "string" ],
> "default" : null
> }, {
> "name" : "last",
> "type" : [ "null", "string" ],
> "default" : null
> } ]
> }
> {noformat}
>
> As you can see, there's no default values for fields. Would you please tell
> me how to obtain such values?
> Thank you.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-248) make unions a named type
[
https://issues.apache.org/jira/browse/AVRO-248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089555#comment-17089555
]
Andy Le commented on AVRO-248:
--
[~teabot] I think it's a good time to have RFCs for Avro 2.x
> make unions a named type
>
>
> Key: AVRO-248
> URL: https://issues.apache.org/jira/browse/AVRO-248
> Project: Apache Avro
> Issue Type: New Feature
> Components: spec
>Reporter: Doug Cutting
>Priority: Major
>
> Unions are currently anonymous. However it might be convenient if they were
> named. In particular:
> - when code is generated for a union, a class could be generated that
> includes an enum indicating which branch of the union is taken, e.g., a union
> of string and int named Foo might cause a Java class like {code}
> public class Foo {
> public static enum Type {STRING, INT};
> private Type type;
> private Object datum;
> public Type getType();
> public String getString() { if (type==STRING) return (String)datum; else
> throw ... }
> public void setString(String s) { type = STRING; datum = s; }
>
> }
> {code} Then Java applications can easily use a switch statement to process
> union values rather than using instanceof.
> - when using reflection, an abstract class with a set of concrete
> implementations can be represented as a union (AVRO-241). However, if one
> wishes to create an array one must know the name of the base class, which is
> not represented in the Avro schema. One approach would be to add an
> annotation to the reflected array schema (AVRO-242) noting the base class.
> But if the union itself were named, that could name the base class. This
> would also make reflected protocol interfaces more consise, since the base
> class name could be used in parameters return types and fields.
> - Generalizing the above: Avro lacks class inheritance, unions are a way to
> model inheritance, and this model is more useful if the union is named.
> This would be an incompatible change to schemas. If we go this way, we
> should probably rename 1.3 to 2.0. Note that AVRO-160 proposes an
> incompatible change to data file formats, which may also force a major
> release.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2723) Avro Java: Obtaining default field values for POJO objects with ReflectData
[
https://issues.apache.org/jira/browse/AVRO-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089264#comment-17089264
]
Andy Le commented on AVRO-2723:
---
[~cutting] Sorry for this late reply
> Is there a reason not to use the default value from the Java class as the
> default value in the schema?
No actually. But the developer experience is much better (nice to have)
> Would this break existing applications?
May be. (Although I ran `mvn test` successfully). I suggest we should gather
RFC for Avro 2.x to include this feature.
> Avro Java: Obtaining default field values for POJO objects with ReflectData
> ---
>
> Key: AVRO-2723
> URL: https://issues.apache.org/jira/browse/AVRO-2723
> Project: Apache Avro
> Issue Type: New Feature
> Components: java
>Affects Versions: 1.9.1
>Reporter: Andy Le
>Priority: Critical
> Attachments: Screen Shot 2020-03-08 at 16.13.29.png
>
>
> Hi guys,
>
> I've got a simple app using Avro Reflection:
>
> {code:java}
> public class App {
> public static void main(String[] args) {
> testReflection();
> }
> static class User {
> public String first = "Andy";
> public String last = "Le";
> }
> static void testReflection(){
> // get the reflected schema for packets
> Schema schema = ReflectData.AllowNull.get().getSchema(User.class);
> System.out.println(schema.toString(true));
> }
> {code}
> The output on console will be:
> {noformat}
> {
> "type" : "record",
> "name" : "User",
> "namespace" : "App",
> "fields" : [ {
> "name" : "first",
> "type" : [ "null", "string" ],
> "default" : null
> }, {
> "name" : "last",
> "type" : [ "null", "string" ],
> "default" : null
> } ]
> }
> {noformat}
>
> As you can see, there's no default values for fields. Would you please tell
> me how to obtain such values?
> Thank you.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2805) Reflection-based schema not loading types
[
https://issues.apache.org/jira/browse/AVRO-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17087023#comment-17087023
]
Andy Le commented on AVRO-2805:
---
[~pcless] Yep. Be strong!
> Reflection-based schema not loading types
> --
>
> Key: AVRO-2805
> URL: https://issues.apache.org/jira/browse/AVRO-2805
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.1
>Reporter: Pedro Cardoso Silva
>Priority: Critical
>
> Avro reflection is unable to generate a schema for the following definition:
> {code:java}
> public class Definition {
> public Map
> }
> public enum Type {
> A,
> B,
> C
> }
> {code}
> {code:java}
> // Test code
> Schema schema = ReflectData.get().getSchema(Definition.class)
> {code}
> Fails with:
> Undefined name: "FieldType"
> org.apache.avro.SchemaParseException: Undefined name: "Type"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2808) Java: ReflectData incorrectly handles hidden fields
[
https://issues.apache.org/jira/browse/AVRO-2808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17087022#comment-17087022
]
Andy Le commented on AVRO-2808:
---
I think we can ignore fields `this$0`. Should I make a PR against this issue?
> Java: ReflectData incorrectly handles hidden fields
> ---
>
> Key: AVRO-2808
> URL: https://issues.apache.org/jira/browse/AVRO-2808
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.2
>Reporter: Andy Le
>Priority: Major
>
> Hi guys,
> I've got the following test:
> {code:java}
> public ReflectTest{
> public class Definition {
> public Map tokens;
> }
> public enum Type {
> A,
> B,
> C
> }
> @Test
> public void testAvro2805() {
> Schema schema = ReflectData.get().getSchema(Definition.class);
> final String schemaString = schema.toString(true);
> System.out.println(schemaString);
> }
> }
> {code}
> When I ran the test, an exception is raised:
> {noformat}
> org.apache.avro.SchemaParseException: Illegal character in: this$0
> at org.apache.avro.Schema.validateName(Schema.java:1530)
> at org.apache.avro.Schema.access$400(Schema.java:87)
> at org.apache.avro.Schema$Field.(Schema.java:518)
> at org.apache.avro.Schema$Field.(Schema.java:557)
> at
> org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:683)
> at
> org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:335)
> at
> org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:332)
> {noformat}
> From the log, you may see that: we should not handle Java hidden field
> `this$0` which is used to hold reference to outer class instances.
> This issue is somehow related to [this
> one|https://issues.apache.org/jira/browse/AVRO-2805]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2775) JacksonUtils: exception when calling toJsonNode()
[
https://issues.apache.org/jira/browse/AVRO-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17086919#comment-17086919
]
Andy Le commented on AVRO-2775:
---
[~cutting] any other feedback?
> JacksonUtils: exception when calling toJsonNode()
> --
>
> Key: AVRO-2775
> URL: https://issues.apache.org/jira/browse/AVRO-2775
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.2
>Reporter: Andy Le
>Priority: Major
>
> I've got a simple test as followed
> {code:java}
> public class TestJacksonUtils {
> public static class Age{
> public int value = 9;
> }
> @Test
> public void testToJson(){
> Map kv = new HashMap<>();
> kv.put("age", 9);
> JsonNode node1 = JacksonUtils.toJsonNode(kv); // -> This is OK
> Object obj = new Age();
> JsonNode node2 = JacksonUtils.toJsonNode(obj); // -> This will trigger an
> exception
> }
> }
> {code}
> When I ran the test:
> {noformat}
> org.apache.avro.AvroRuntimeException: Unknown datum class: class
> org.apache.avro.util.internal.TestJacksonUtils$Age
> at
> org.apache.avro.util.internal.JacksonUtils.toJson(JacksonUtils.java:87)
> at
> org.apache.avro.util.internal.JacksonUtils.toJsonNode(JacksonUtils.java:48)
> at
> org.apache.avro.util.internal.TestJacksonUtils.testToJson(TestJacksonUtils.java:20)
> {noformat}
> I've read the code & tests for JacksonUtils. Instead of raising exceptions at
> [line
> #87|https://github.com/apache/avro/blob/master/lang/java/avro/src/main/java/org/apache/avro/util/internal/JacksonUtils.java#L87],
> I see we can auto convert objects into maps, every thing's gonna fine.
> My question is:
> - Is raising exception acceptable?
> - Any other way to have `toJsonNode` for general objects?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (AVRO-2808) Java: ReflectData incorrectly handles hidden fields
Andy Le created AVRO-2808:
-
Summary: Java: ReflectData incorrectly handles hidden fields
Key: AVRO-2808
URL: https://issues.apache.org/jira/browse/AVRO-2808
Project: Apache Avro
Issue Type: Bug
Components: java
Reporter: Andy Le
Hi guys,
I've got the following test:
{code:java}
public ReflectTest{
public class Definition {
public Map tokens;
}
public enum Type {
A,
B,
C
}
@Test
public void testAvro2805() {
Schema schema = ReflectData.get().getSchema(Definition.class);
final String schemaString = schema.toString(true);
System.out.println(schemaString);
}
}
{code}
When I ran the test, an exception is raised:
{noformat}
org.apache.avro.SchemaParseException: Illegal character in: this$0
at org.apache.avro.Schema.validateName(Schema.java:1530)
at org.apache.avro.Schema.access$400(Schema.java:87)
at org.apache.avro.Schema$Field.(Schema.java:518)
at org.apache.avro.Schema$Field.(Schema.java:557)
at
org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:683)
at
org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:335)
at
org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:332)
{noformat}
>From the log, you may see that: we should not handle Java hidden field
>`this$0` which is used to hold reference to outer class instances.
This issue is somehow related to [this
one|https://issues.apache.org/jira/browse/AVRO-2805]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (AVRO-2808) Java: ReflectData incorrectly handles hidden fields
[
https://issues.apache.org/jira/browse/AVRO-2808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Le updated AVRO-2808:
--
Affects Version/s: 1.9.2
> Java: ReflectData incorrectly handles hidden fields
> ---
>
> Key: AVRO-2808
> URL: https://issues.apache.org/jira/browse/AVRO-2808
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.2
>Reporter: Andy Le
>Priority: Major
>
> Hi guys,
> I've got the following test:
> {code:java}
> public ReflectTest{
> public class Definition {
> public Map tokens;
> }
> public enum Type {
> A,
> B,
> C
> }
> @Test
> public void testAvro2805() {
> Schema schema = ReflectData.get().getSchema(Definition.class);
> final String schemaString = schema.toString(true);
> System.out.println(schemaString);
> }
> }
> {code}
> When I ran the test, an exception is raised:
> {noformat}
> org.apache.avro.SchemaParseException: Illegal character in: this$0
> at org.apache.avro.Schema.validateName(Schema.java:1530)
> at org.apache.avro.Schema.access$400(Schema.java:87)
> at org.apache.avro.Schema$Field.(Schema.java:518)
> at org.apache.avro.Schema$Field.(Schema.java:557)
> at
> org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:683)
> at
> org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:335)
> at
> org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:332)
> {noformat}
> From the log, you may see that: we should not handle Java hidden field
> `this$0` which is used to hold reference to outer class instances.
> This issue is somehow related to [this
> one|https://issues.apache.org/jira/browse/AVRO-2805]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2808) Java: ReflectData incorrectly handles hidden fields
[
https://issues.apache.org/jira/browse/AVRO-2808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17086918#comment-17086918
]
Andy Le commented on AVRO-2808:
---
[~cutting] [~sekikn] PATL
> Java: ReflectData incorrectly handles hidden fields
> ---
>
> Key: AVRO-2808
> URL: https://issues.apache.org/jira/browse/AVRO-2808
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.2
>Reporter: Andy Le
>Priority: Major
>
> Hi guys,
> I've got the following test:
> {code:java}
> public ReflectTest{
> public class Definition {
> public Map tokens;
> }
> public enum Type {
> A,
> B,
> C
> }
> @Test
> public void testAvro2805() {
> Schema schema = ReflectData.get().getSchema(Definition.class);
> final String schemaString = schema.toString(true);
> System.out.println(schemaString);
> }
> }
> {code}
> When I ran the test, an exception is raised:
> {noformat}
> org.apache.avro.SchemaParseException: Illegal character in: this$0
> at org.apache.avro.Schema.validateName(Schema.java:1530)
> at org.apache.avro.Schema.access$400(Schema.java:87)
> at org.apache.avro.Schema$Field.(Schema.java:518)
> at org.apache.avro.Schema$Field.(Schema.java:557)
> at
> org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:683)
> at
> org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:335)
> at
> org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:332)
> {noformat}
> From the log, you may see that: we should not handle Java hidden field
> `this$0` which is used to hold reference to outer class instances.
> This issue is somehow related to [this
> one|https://issues.apache.org/jira/browse/AVRO-2805]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2805) Reflection-based schema not loading types
[
https://issues.apache.org/jira/browse/AVRO-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17086916#comment-17086916
]
Andy Le commented on AVRO-2805:
---
[~pcless] I'm just a contributor :-P Can't close your issue.
But feel free to show your Kotlin snippet. I think there's nothing specific to
Kotlin here.
> Reflection-based schema not loading types
> --
>
> Key: AVRO-2805
> URL: https://issues.apache.org/jira/browse/AVRO-2805
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.1
>Reporter: Pedro Cardoso Silva
>Priority: Critical
>
> Avro reflection is unable to generate a schema for the following definition:
> {code:java}
> public class Definition {
> public Map
> }
> public enum Type {
> A,
> B,
> C
> }
> {code}
> {code:java}
> // Test code
> Schema schema = ReflectData.get().getSchema(Definition.class)
> {code}
> Fails with:
> Undefined name: "FieldType"
> org.apache.avro.SchemaParseException: Undefined name: "Type"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2805) Reflection-based schema not loading types
[
https://issues.apache.org/jira/browse/AVRO-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17086869#comment-17086869
]
Andy Le commented on AVRO-2805:
---
[~pcless] Your example code contain errors. I decided to wrap your logic inside
a test
{code:java}
public static class Definition {
public Map tokens;
}
public enum Type {
A,
B,
C
}
@Test
public void testAvro2805() {
Schema schema = ReflectData.get().getSchema(Definition.class);
final String schemaString = schema.toString(true);
System.out.println(schemaString);
}
{code}
I can NOT see your exception above. I've got this one instead:
{noformat}
org.apache.avro.SchemaParseException: Illegal character in: this$0
at org.apache.avro.Schema.validateName(Schema.java:1530)
at org.apache.avro.Schema.access$400(Schema.java:87)
at org.apache.avro.Schema$Field.(Schema.java:518)
at org.apache.avro.Schema$Field.(Schema.java:557)
at
org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:683)
{noformat}
Please kindly verify your issue again and let me know.
> Reflection-based schema not loading types
> --
>
> Key: AVRO-2805
> URL: https://issues.apache.org/jira/browse/AVRO-2805
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.1
>Reporter: Pedro Cardoso Silva
>Priority: Critical
>
> Avro reflection is unable to generate a schema for the following definition:
> {code:java}
> public class Definition {
> pubilc Map
> }
> public enum Type {
> A,
> B,
> C
> }
> {code}
> {code:java}
> // Test code
> Schema schema = ReflectData.get().getSchema(Definition.class)
> {code}
> Fails with:
> Undefined name: "FieldType"
> org.apache.avro.SchemaParseException: Undefined name: "Type"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2775) JacksonUtils: exception when calling toJsonNode()
[
https://issues.apache.org/jira/browse/AVRO-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081209#comment-17081209
]
Andy Le commented on AVRO-2775:
---
[~cutting]
> Is it incompatible enough to worry about?
Do you think our current tests are enough to validate compatibility? On our
mailing lists, I see so many concerns about backward compatibility. Do you
think we need to make breaking changes with Avro 2.x ?
> JacksonUtils: exception when calling toJsonNode()
> --
>
> Key: AVRO-2775
> URL: https://issues.apache.org/jira/browse/AVRO-2775
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.2
>Reporter: Andy Le
>Priority: Major
>
> I've got a simple test as followed
> {code:java}
> public class TestJacksonUtils {
> public static class Age{
> public int value = 9;
> }
> @Test
> public void testToJson(){
> Map kv = new HashMap<>();
> kv.put("age", 9);
> JsonNode node1 = JacksonUtils.toJsonNode(kv); // -> This is OK
> Object obj = new Age();
> JsonNode node2 = JacksonUtils.toJsonNode(obj); // -> This will trigger an
> exception
> }
> }
> {code}
> When I ran the test:
> {noformat}
> org.apache.avro.AvroRuntimeException: Unknown datum class: class
> org.apache.avro.util.internal.TestJacksonUtils$Age
> at
> org.apache.avro.util.internal.JacksonUtils.toJson(JacksonUtils.java:87)
> at
> org.apache.avro.util.internal.JacksonUtils.toJsonNode(JacksonUtils.java:48)
> at
> org.apache.avro.util.internal.TestJacksonUtils.testToJson(TestJacksonUtils.java:20)
> {noformat}
> I've read the code & tests for JacksonUtils. Instead of raising exceptions at
> [line
> #87|https://github.com/apache/avro/blob/master/lang/java/avro/src/main/java/org/apache/avro/util/internal/JacksonUtils.java#L87],
> I see we can auto convert objects into maps, every thing's gonna fine.
> My question is:
> - Is raising exception acceptable?
> - Any other way to have `toJsonNode` for general objects?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2792) Fix C# logical type tests to work in other timezones than UTC
[
https://issues.apache.org/jira/browse/AVRO-2792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080548#comment-17080548
]
Andy Le commented on AVRO-2792:
---
[~sekikn]
> Avro's timestamp logical type has a long value that represents the difference
> from the unix epoch (1 Jan 1970 00:00:00 UTC) and it doesn't have timezone
> information.
Do you think that we need to update the Spec to have timezone information?
> Fix C# logical type tests to work in other timezones than UTC
> -
>
> Key: AVRO-2792
> URL: https://issues.apache.org/jira/browse/AVRO-2792
> Project: Apache Avro
> Issue Type: Bug
>Reporter: Kengo Seki
>Assignee: Kengo Seki
>Priority: Major
>
> I ran the C# unit tests outside of a container and got the following error.
> This is because my machine's timezone is JST (UTC+9).
> {code}
> $ cd lang/csharp
> $ ./build.sh test
> (snip)
> X TestLogical_TimestampMicrosecond() [22ms]
> Error Message:
> Expected: 1990-01-01 14:15:30
> But was: 1990-01-01 05:15:30
> Stack Trace:
> at Avro.Test.Generic.GenericTests.test[T](String s, T value) in
> /home/sekikn/repos/avro/lang/csharp/src/apache/test/Generic/GenericTests.cs:line
> 38
>at Avro.Test.Generic.GenericTests.TestLogical_TimestampMicrosecond() in
> /home/sekikn/repos/avro/lang/csharp/src/apache/test/Generic/GenericTests.cs:line
> 139
> X TestLogical_TimestampMillisecond() [< 1ms]
> Error Message:
> Expected: 1990-01-01 14:15:30
> But was: 1990-01-01 05:15:30
> Stack Trace:
> at Avro.Test.Generic.GenericTests.test[T](String s, T value) in
> /home/sekikn/repos/avro/lang/csharp/src/apache/test/Generic/GenericTests.cs:line
> 38
>at Avro.Test.Generic.GenericTests.TestLogical_TimestampMillisecond() in
> /home/sekikn/repos/avro/lang/csharp/src/apache/test/Generic/GenericTests.cs:line
> 133
> X TestTimestampMicrosecond("01/01/2019 14:20:00","01/01/2019 14:20:00Z") [<
> 1ms]
> Error Message:
> Expected: 2019-01-01 14:20:00
> But was: 2019-01-01 05:20:00
> Stack Trace:
> at Avro.Test.LogicalTypeTests.TestTimestampMicrosecond(String s, String
> e) in
> /home/sekikn/repos/avro/lang/csharp/src/apache/test/Util/LogicalTypeTests.cs:line
> 143
> X TestTimestampMillisecond("01/01/2019 14:20:00","01/01/2019 14:20:00Z") [<
> 1ms]
> Error Message:
> Expected: 2019-01-01 14:20:00
> But was: 2019-01-01 05:20:00
> Stack Trace:
> at Avro.Test.LogicalTypeTests.TestTimestampMillisecond(String s, String
> e) in
> /home/sekikn/repos/avro/lang/csharp/src/apache/test/Util/LogicalTypeTests.cs:line
> 119
> Test Run Failed.
> Total tests: 584
> Passed: 580
> Failed: 4
> Total time: 2.3598 Seconds
> {code}
> [As the specification
> says|https://avro.apache.org/docs/current/spec.html#Timestamp+%28millisecond+precision%29],
> Avro's timestamp logical type has a long value that represents the
> difference from the unix epoch (1 Jan 1970 00:00:00 UTC) and it doesn't have
> timezone information. So when deserializing it, it's returned as a C#
> DateTime object with UTC.
> Therefore, if the input string lacks timezone information, we should assume
> it represents UTC datetime in these test cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2775) JacksonUtils: exception when calling toJsonNode()
[
https://issues.apache.org/jira/browse/AVRO-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080173#comment-17080173
]
Andy Le commented on AVRO-2775:
---
[~cutting]
> changing this would permit them to include arbitrary Java data structures
It's correct. Such structures may be represented as Maps
> which might obscure a bug if this was not intended
Would you elaborate more?
> JacksonUtils: exception when calling toJsonNode()
> --
>
> Key: AVRO-2775
> URL: https://issues.apache.org/jira/browse/AVRO-2775
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.2
>Reporter: Andy Le
>Priority: Major
>
> I've got a simple test as followed
> {code:java}
> public class TestJacksonUtils {
> public static class Age{
> public int value = 9;
> }
> @Test
> public void testToJson(){
> Map kv = new HashMap<>();
> kv.put("age", 9);
> JsonNode node1 = JacksonUtils.toJsonNode(kv); // -> This is OK
> Object obj = new Age();
> JsonNode node2 = JacksonUtils.toJsonNode(obj); // -> This will trigger an
> exception
> }
> }
> {code}
> When I ran the test:
> {noformat}
> org.apache.avro.AvroRuntimeException: Unknown datum class: class
> org.apache.avro.util.internal.TestJacksonUtils$Age
> at
> org.apache.avro.util.internal.JacksonUtils.toJson(JacksonUtils.java:87)
> at
> org.apache.avro.util.internal.JacksonUtils.toJsonNode(JacksonUtils.java:48)
> at
> org.apache.avro.util.internal.TestJacksonUtils.testToJson(TestJacksonUtils.java:20)
> {noformat}
> I've read the code & tests for JacksonUtils. Instead of raising exceptions at
> [line
> #87|https://github.com/apache/avro/blob/master/lang/java/avro/src/main/java/org/apache/avro/util/internal/JacksonUtils.java#L87],
> I see we can auto convert objects into maps, every thing's gonna fine.
> My question is:
> - Is raising exception acceptable?
> - Any other way to have `toJsonNode` for general objects?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (AVRO-2785) Update specs on how unions encoded
[ https://issues.apache.org/jira/browse/AVRO-2785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Le updated AVRO-2785: -- Summary: Update specs on how unions encoded (was: Update how unions encoded) > Update specs on how unions encoded > -- > > Key: AVRO-2785 > URL: https://issues.apache.org/jira/browse/AVRO-2785 > Project: Apache Avro > Issue Type: Improvement > Components: doc, spec >Affects Versions: 1.9.2 >Reporter: Andy Le >Priority: Major > > h1. Overview > This issue is associated with [our mailing > discussions|https://lists.apache.org/thread.html/r3f4b8b40ec9604e4f3854fb59f9219db9bbb58eebea4b9d2aa097688%40%3Cuser.avro.apache.org%3E] > h1. Solution > Update the Spec -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AVRO-2785) Update how unions encoded
[ https://issues.apache.org/jira/browse/AVRO-2785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Le updated AVRO-2785: -- Summary: Update how unions encoded (was: Update how union encoded) > Update how unions encoded > - > > Key: AVRO-2785 > URL: https://issues.apache.org/jira/browse/AVRO-2785 > Project: Apache Avro > Issue Type: Improvement > Components: doc, spec >Affects Versions: 1.9.2 >Reporter: Andy Le >Priority: Major > > h1. Overview > This issue is associated with [our mailing > discussions|https://lists.apache.org/thread.html/r3f4b8b40ec9604e4f3854fb59f9219db9bbb58eebea4b9d2aa097688%40%3Cuser.avro.apache.org%3E] > h1. Solution > Update the Spec -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (AVRO-2785) Update how union encoded
Andy Le created AVRO-2785: - Summary: Update how union encoded Key: AVRO-2785 URL: https://issues.apache.org/jira/browse/AVRO-2785 Project: Apache Avro Issue Type: Improvement Components: doc, spec Affects Versions: 1.9.2 Reporter: Andy Le h1. Overview This issue is associated with [our mailing discussions|https://lists.apache.org/thread.html/r3f4b8b40ec9604e4f3854fb59f9219db9bbb58eebea4b9d2aa097688%40%3Cuser.avro.apache.org%3E] h1. Solution Update the Spec -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AVRO-2769) After upgrade to 1.9.2, generated code for "put" uses value and value$
[
https://issues.apache.org/jira/browse/AVRO-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17064834#comment-17064834
]
Andy Le commented on AVRO-2769:
---
[~alexanderUV] would you please share the detail (esp the schema) ?
> After upgrade to 1.9.2, generated code for "put" uses value and value$
> --
>
> Key: AVRO-2769
> URL: https://issues.apache.org/jira/browse/AVRO-2769
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.2
> Environment: macOS Catalina
> JDK 8 (oracle)
>Reporter: Alexander Ubillus
>Priority: Blocker
>
> After upgrade to 1.9.2, generated code for "put" uses value and value$,
> causing compilation exceptions:
> {code:java}
> // Used by DatumReader. Applications should not call.
> @SuppressWarnings(value="unchecked")
> public void put(int field$, java.lang.Object value$) {
> switch (field$) {
> case 0: name = (java.lang.CharSequence)value; break;
> case 1: length = (java.math.BigDecimal)value; break;
> case 2: metadata =
> (java.util.Map)value; break;
> default: throw new org.apache.avro.AvroRuntimeException("Bad index");
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (AVRO-2778) Java: Reflect raises exceptions while dealing with non-static classes
Andy Le created AVRO-2778:
-
Summary: Java: Reflect raises exceptions while dealing with
non-static classes
Key: AVRO-2778
URL: https://issues.apache.org/jira/browse/AVRO-2778
Project: Apache Avro
Issue Type: Bug
Components: java
Affects Versions: 1.9.2
Reporter: Andy Le
Hi guys,
I've got the following simple code:
{code:java}
public class TestReflect {
public class Human{
String name = "Andy";
}
@Test
public void testNonStaticClasses(){
ReflectData.get().getSchema(Human.class) ;
}
}
{code}
When I run the test, an exception is spawn:
{noformat}
org.apache.avro.SchemaParseException: Illegal character in: this$0
at org.apache.avro.Schema.validateName(Schema.java:1530)
at org.apache.avro.Schema.access$400(Schema.java:87)
at org.apache.avro.Schema$Field.(Schema.java:518)
at org.apache.avro.Schema$Field.(Schema.java:557)
at
org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:645)
at
org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:335)
at
org.apache.avro.specific.SpecificData$3.computeValue(SpecificData.java:1)
at java.lang.ClassValue.getFromHashMap(ClassValue.java:227)
at java.lang.ClassValue.getFromBackup(ClassValue.java:209)
at java.lang.ClassValue.get(ClassValue.java:115)
at
org.apache.avro.specific.SpecificData.getSchema(SpecificData.java:346)
...
{noformat}
It seems to me that: Avro Reflect does NOT effectively handle non-static
classes.
Should I make a PR against this issue?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2647) specification does not fully specify semantics for unions in default types
[
https://issues.apache.org/jira/browse/AVRO-2647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062145#comment-17062145
]
Andy Le commented on AVRO-2647:
---
[~rogpeppe] [~rskraba] Can we just throw errors while encountering such
ambiguous schema?
> specification does not fully specify semantics for unions in default types
> --
>
> Key: AVRO-2647
> URL: https://issues.apache.org/jira/browse/AVRO-2647
> Project: Apache Avro
> Issue Type: Bug
> Components: doc
>Reporter: Roger
>Priority: Minor
>
> Currently the specification does not make the semantics clear for union types
> within complex types clear. In particular, the spec talks about union fields,
> but leaves the semantics for unions in other contexts unspecified.
> Here's an example which is undefined according to the current specification:
> {code:json}
> {
> "type": "record",
> "name": "R",
> "fields": [
> {
> "name": "F",
> "type": {
> "type": "array",
> "items": [
> {
> "type": "enum",
> "name": "E1",
> "symbols": ["A", "B"]
> },
> {
> "type": "enum",
> "name": "E2",
> "symbols": ["B", "A", "C"]
> }
> ]
> },
> "default": ["A", "B", "C"]
> }
> ]
> }
> {code}
> By experiment, most implementations seem to have chosen the semantics that
> are documented in this PR.
> In Java, the schema above is parsed without error, but when attempting to use
> the default value, it fails with a NullPointerException (trying to find the
> symbol C in E1). (Thanks for Ryan Skraba for this).
> In [gogen-avro|https://github.com/actgardner/gogen-avro] it generates invalid
> code because it's assuming E1 but generating the symbol for "C" anyway.
> FWIW at some point in the future, I believe that it would be nice to align
> the default value specification with the JSON encoding for Avro so there
> aren't two subtly different JSON encodings of an Avro value.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2775) JacksonUtils: exception when calling toJsonNode()
[
https://issues.apache.org/jira/browse/AVRO-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17061014#comment-17061014
]
Andy Le commented on AVRO-2775:
---
I also think we should refactor JacksonUtils to clean it a little bit.
For example, currently we have:
{code:java}
public static Object toObject(JsonNode jsonNode, Schema schema) {
if (schema != null && schema.getType().equals(Schema.Type.UNION)) {
return toObject(jsonNode, schema.getTypes().get(0));
}
if (jsonNode == null) {
return null;
} else if (jsonNode.isNull()) {
return JsonProperties.NULL_VALUE;
} else if (jsonNode.isBoolean()) {
return jsonNode.asBoolean();
}
//. ..
}
{code}
We can have a better code the above:
{code:java}
public static Object toObject(JsonNode jsonNode, Schema schema) {
if (schema != null && schema.getType().equals(Schema.Type.UNION)) {
return toObject(jsonNode, schema.getTypes().get(0));
}
if (jsonNode == null) {
return null;
}
if (jsonNode.isNull()) {
return JsonProperties.NULL_VALUE;
}
if (jsonNode.isBoolean()) {
return jsonNode.asBoolean();
}
// ..
}
{code}
[~sekikn] do you think it's worth to refactor such code?
> JacksonUtils: exception when calling toJsonNode()
> --
>
> Key: AVRO-2775
> URL: https://issues.apache.org/jira/browse/AVRO-2775
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.2
>Reporter: Andy Le
>Priority: Major
>
> I've got a simple test as followed
> {code:java}
> public class TestJacksonUtils {
> public static class Age{
> public int value = 9;
> }
> @Test
> public void testToJson(){
> Map kv = new HashMap<>();
> kv.put("age", 9);
> JsonNode node1 = JacksonUtils.toJsonNode(kv); // -> This is OK
> Object obj = new Age();
> JsonNode node2 = JacksonUtils.toJsonNode(obj); // -> This will trigger an
> exception
> }
> }
> {code}
> When I ran the test:
> {noformat}
> org.apache.avro.AvroRuntimeException: Unknown datum class: class
> org.apache.avro.util.internal.TestJacksonUtils$Age
> at
> org.apache.avro.util.internal.JacksonUtils.toJson(JacksonUtils.java:87)
> at
> org.apache.avro.util.internal.JacksonUtils.toJsonNode(JacksonUtils.java:48)
> at
> org.apache.avro.util.internal.TestJacksonUtils.testToJson(TestJacksonUtils.java:20)
> {noformat}
> I've read the code & tests for JacksonUtils. Instead of raising exceptions at
> [line
> #87|https://github.com/apache/avro/blob/master/lang/java/avro/src/main/java/org/apache/avro/util/internal/JacksonUtils.java#L87],
> I see we can auto convert objects into maps, every thing's gonna fine.
> My question is:
> - Is raising exception acceptable?
> - Any other way to have `toJsonNode` for general objects?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2775) JacksonUtils: exception when calling toJsonNode()
[
https://issues.apache.org/jira/browse/AVRO-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17060589#comment-17060589
]
Andy Le commented on AVRO-2775:
---
Another problem is: currently JacksonUtils does NOT handle Short and Byte (see
[this
code|https://github.com/apache/avro/blob/master/lang/java/avro/src/main/java/org/apache/avro/util/internal/JacksonUtils.java#L82-L84]).
So I'm thinking of creating a PR to address them all.
[~fokko] [~sekikn] Please review my issue. Thank you!
> JacksonUtils: exception when calling toJsonNode()
> --
>
> Key: AVRO-2775
> URL: https://issues.apache.org/jira/browse/AVRO-2775
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.2
>Reporter: Andy Le
>Priority: Major
>
> I've got a simple test as followed
> {code:java}
> public class TestJacksonUtils {
> public static class Age{
> public int value = 9;
> }
> @Test
> public void testToJson(){
> Map kv = new HashMap<>();
> kv.put("age", 9);
> JsonNode node1 = JacksonUtils.toJsonNode(kv); // -> This is OK
> Object obj = new Age();
> JsonNode node2 = JacksonUtils.toJsonNode(obj); // -> This will trigger an
> exception
> }
> }
> {code}
> When I ran the test:
> {noformat}
> org.apache.avro.AvroRuntimeException: Unknown datum class: class
> org.apache.avro.util.internal.TestJacksonUtils$Age
> at
> org.apache.avro.util.internal.JacksonUtils.toJson(JacksonUtils.java:87)
> at
> org.apache.avro.util.internal.JacksonUtils.toJsonNode(JacksonUtils.java:48)
> at
> org.apache.avro.util.internal.TestJacksonUtils.testToJson(TestJacksonUtils.java:20)
> {noformat}
> I've read the code & tests for JacksonUtils. Instead of raising exceptions at
> [line
> #87|https://github.com/apache/avro/blob/master/lang/java/avro/src/main/java/org/apache/avro/util/internal/JacksonUtils.java#L87],
> I see we can auto convert objects into maps, every thing's gonna fine.
> My question is:
> - Is raising exception acceptable?
> - Any other way to have `toJsonNode` for general objects?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (AVRO-2775) JacksonUtils: exception when calling toJsonNode()
Andy Le created AVRO-2775:
-
Summary: JacksonUtils: exception when calling toJsonNode()
Key: AVRO-2775
URL: https://issues.apache.org/jira/browse/AVRO-2775
Project: Apache Avro
Issue Type: Bug
Components: java
Affects Versions: 1.9.2
Reporter: Andy Le
I've got a simple test as followed
{code:java}
public class TestJacksonUtils {
public static class Age{
public int value = 9;
}
@Test
public void testToJson(){
Map kv = new HashMap<>();
kv.put("age", 9);
JsonNode node1 = JacksonUtils.toJsonNode(kv); // -> This is OK
Object obj = new Age();
JsonNode node2 = JacksonUtils.toJsonNode(obj); // -> This will trigger an
exception
}
}
{code}
When I ran the test:
{noformat}
org.apache.avro.AvroRuntimeException: Unknown datum class: class
org.apache.avro.util.internal.TestJacksonUtils$Age
at
org.apache.avro.util.internal.JacksonUtils.toJson(JacksonUtils.java:87)
at
org.apache.avro.util.internal.JacksonUtils.toJsonNode(JacksonUtils.java:48)
at
org.apache.avro.util.internal.TestJacksonUtils.testToJson(TestJacksonUtils.java:20)
{noformat}
I've read the code & tests for JacksonUtils. Instead of raising exceptions at
[line
#87|https://github.com/apache/avro/blob/master/lang/java/avro/src/main/java/org/apache/avro/util/internal/JacksonUtils.java#L87],
I see we can auto convert objects into maps, every thing's gonna fine.
My question is:
- Is raising exception acceptable?
- Any other way to have `toJsonNode` for general objects?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2723) Avro Java: Obtaining default field values for POJO objects with ReflectData
[
https://issues.apache.org/jira/browse/AVRO-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17054694#comment-17054694
]
Andy Le commented on AVRO-2723:
---
[~sekikn]
> So UseInitialValueAsDefault or something might be more precise?
Fine for me
> didn't see them when I ran mvn javadoc:aggregate in a Docker container,
> though many warnings were shown as follows. Maybe you're using Java 11?...
My fault. I directly run the command without being inside the Docker
environment.
> Avro Java: Obtaining default field values for POJO objects with ReflectData
> ---
>
> Key: AVRO-2723
> URL: https://issues.apache.org/jira/browse/AVRO-2723
> Project: Apache Avro
> Issue Type: New Feature
> Components: java
>Affects Versions: 1.9.1
>Reporter: Andy Le
>Priority: Critical
> Attachments: Screen Shot 2020-03-08 at 16.13.29.png
>
>
> Hi guys,
>
> I've got a simple app using Avro Reflection:
>
> {code:java}
> public class App {
> public static void main(String[] args) {
> testReflection();
> }
> static class User {
> public String first = "Andy";
> public String last = "Le";
> }
> static void testReflection(){
> // get the reflected schema for packets
> Schema schema = ReflectData.AllowNull.get().getSchema(User.class);
> System.out.println(schema.toString(true));
> }
> {code}
> The output on console will be:
> {noformat}
> {
> "type" : "record",
> "name" : "User",
> "namespace" : "App",
> "fields" : [ {
> "name" : "first",
> "type" : [ "null", "string" ],
> "default" : null
> }, {
> "name" : "last",
> "type" : [ "null", "string" ],
> "default" : null
> } ]
> }
> {noformat}
>
> As you can see, there's no default values for fields. Would you please tell
> me how to obtain such values?
> Thank you.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (AVRO-2723) Avro Java: Obtaining default field values for POJO objects with ReflectData
[
https://issues.apache.org/jira/browse/AVRO-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Le updated AVRO-2723:
--
Summary: Avro Java: Obtaining default field values for POJO objects with
ReflectData (was: Avro Java: Obtaining default values for POJO objects with
ReflectData)
> Avro Java: Obtaining default field values for POJO objects with ReflectData
> ---
>
> Key: AVRO-2723
> URL: https://issues.apache.org/jira/browse/AVRO-2723
> Project: Apache Avro
> Issue Type: New Feature
> Components: java
>Affects Versions: 1.9.1
>Reporter: Andy Le
>Priority: Critical
> Attachments: Screen Shot 2020-03-08 at 16.13.29.png
>
>
> Hi guys,
>
> I've got a simple app using Avro Reflection:
>
> {code:java}
> public class App {
> public static void main(String[] args) {
> testReflection();
> }
> static class User {
> public String first = "Andy";
> public String last = "Le";
> }
> static void testReflection(){
> // get the reflected schema for packets
> Schema schema = ReflectData.AllowNull.get().getSchema(User.class);
> System.out.println(schema.toString(true));
> }
> {code}
> The output on console will be:
> {noformat}
> {
> "type" : "record",
> "name" : "User",
> "namespace" : "App",
> "fields" : [ {
> "name" : "first",
> "type" : [ "null", "string" ],
> "default" : null
> }, {
> "name" : "last",
> "type" : [ "null", "string" ],
> "default" : null
> } ]
> }
> {noformat}
>
> As you can see, there's no default values for fields. Would you please tell
> me how to obtain such values?
> Thank you.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2723) Avro Java: Obtaining default values for POJO objects with ReflectData
[
https://issues.apache.org/jira/browse/AVRO-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17054340#comment-17054340
]
Andy Le commented on AVRO-2723:
---
[~rskraba] [~sekikn] I follow [this
guideline|https://cwiki.apache.org/confluence/display/AVRO/How+To+Contribute]
to contribute code. But when I issue commmand
{code:bash}
$ mvn javadoc:aggregate
{code}
I see various errors in other classes (not ReflectData). What should I do? Make
another PR to improve the documentation, right?
!Screen Shot 2020-03-08 at 16.13.29.png!
> Avro Java: Obtaining default values for POJO objects with ReflectData
> -
>
> Key: AVRO-2723
> URL: https://issues.apache.org/jira/browse/AVRO-2723
> Project: Apache Avro
> Issue Type: New Feature
> Components: java
>Affects Versions: 1.9.1
>Reporter: Andy Le
>Priority: Critical
> Attachments: Screen Shot 2020-03-08 at 16.13.29.png
>
>
> Hi guys,
>
> I've got a simple app using Avro Reflection:
>
> {code:java}
> public class App {
> public static void main(String[] args) {
> testReflection();
> }
> static class User {
> public String first = "Andy";
> public String last = "Le";
> }
> static void testReflection(){
> // get the reflected schema for packets
> Schema schema = ReflectData.AllowNull.get().getSchema(User.class);
> System.out.println(schema.toString(true));
> }
> {code}
> The output on console will be:
> {noformat}
> {
> "type" : "record",
> "name" : "User",
> "namespace" : "App",
> "fields" : [ {
> "name" : "first",
> "type" : [ "null", "string" ],
> "default" : null
> }, {
> "name" : "last",
> "type" : [ "null", "string" ],
> "default" : null
> } ]
> }
> {noformat}
>
> As you can see, there's no default values for fields. Would you please tell
> me how to obtain such values?
> Thank you.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (AVRO-2723) Avro Java: Obtaining default values for POJO objects with ReflectData
[
https://issues.apache.org/jira/browse/AVRO-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Le updated AVRO-2723:
--
Attachment: Screen Shot 2020-03-08 at 16.13.29.png
> Avro Java: Obtaining default values for POJO objects with ReflectData
> -
>
> Key: AVRO-2723
> URL: https://issues.apache.org/jira/browse/AVRO-2723
> Project: Apache Avro
> Issue Type: New Feature
> Components: java
>Affects Versions: 1.9.1
>Reporter: Andy Le
>Priority: Critical
> Attachments: Screen Shot 2020-03-08 at 16.13.29.png
>
>
> Hi guys,
>
> I've got a simple app using Avro Reflection:
>
> {code:java}
> public class App {
> public static void main(String[] args) {
> testReflection();
> }
> static class User {
> public String first = "Andy";
> public String last = "Le";
> }
> static void testReflection(){
> // get the reflected schema for packets
> Schema schema = ReflectData.AllowNull.get().getSchema(User.class);
> System.out.println(schema.toString(true));
> }
> {code}
> The output on console will be:
> {noformat}
> {
> "type" : "record",
> "name" : "User",
> "namespace" : "App",
> "fields" : [ {
> "name" : "first",
> "type" : [ "null", "string" ],
> "default" : null
> }, {
> "name" : "last",
> "type" : [ "null", "string" ],
> "default" : null
> } ]
> }
> {noformat}
>
> As you can see, there's no default values for fields. Would you please tell
> me how to obtain such values?
> Thank you.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2723) Avro Java: Obtaining default values for POJO objects with ReflectData
[
https://issues.apache.org/jira/browse/AVRO-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17054334#comment-17054334
]
Andy Le commented on AVRO-2723:
---
[~rskraba] [~sekikn] Currently we have class `ReflectData.AllowNulls` declared
[at line
#175|https://github.com/apache/avro/blob/791ec6017b03e3cc79fbf546dd9d79d20d476b4e/lang/java/avro/src/main/java/org/apache/avro/reflect/ReflectData.java#L75].
Regarding this issue, should I add implementation for
`ReflectData.AllowDefaults` (please see class DefaultReflector above) ?
> Avro Java: Obtaining default values for POJO objects with ReflectData
> -
>
> Key: AVRO-2723
> URL: https://issues.apache.org/jira/browse/AVRO-2723
> Project: Apache Avro
> Issue Type: New Feature
> Components: java
>Affects Versions: 1.9.1
>Reporter: Andy Le
>Priority: Critical
>
> Hi guys,
>
> I've got a simple app using Avro Reflection:
>
> {code:java}
> public class App {
> public static void main(String[] args) {
> testReflection();
> }
> static class User {
> public String first = "Andy";
> public String last = "Le";
> }
> static void testReflection(){
> // get the reflected schema for packets
> Schema schema = ReflectData.AllowNull.get().getSchema(User.class);
> System.out.println(schema.toString(true));
> }
> {code}
> The output on console will be:
> {noformat}
> {
> "type" : "record",
> "name" : "User",
> "namespace" : "App",
> "fields" : [ {
> "name" : "first",
> "type" : [ "null", "string" ],
> "default" : null
> }, {
> "name" : "last",
> "type" : [ "null", "string" ],
> "default" : null
> } ]
> }
> {noformat}
>
> As you can see, there's no default values for fields. Would you please tell
> me how to obtain such values?
> Thank you.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2723) Avro Java: Obtaining default values for POJO objects with ReflectData
[
https://issues.apache.org/jira/browse/AVRO-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17054327#comment-17054327
]
Andy Le commented on AVRO-2723:
---
[~sekikn] Thank you. I'll use the suggested name
> Avro Java: Obtaining default values for POJO objects with ReflectData
> -
>
> Key: AVRO-2723
> URL: https://issues.apache.org/jira/browse/AVRO-2723
> Project: Apache Avro
> Issue Type: New Feature
> Components: java
>Affects Versions: 1.9.1
>Reporter: Andy Le
>Priority: Critical
>
> Hi guys,
>
> I've got a simple app using Avro Reflection:
>
> {code:java}
> public class App {
> public static void main(String[] args) {
> testReflection();
> }
> static class User {
> public String first = "Andy";
> public String last = "Le";
> }
> static void testReflection(){
> // get the reflected schema for packets
> Schema schema = ReflectData.AllowNull.get().getSchema(User.class);
> System.out.println(schema.toString(true));
> }
> {code}
> The output on console will be:
> {noformat}
> {
> "type" : "record",
> "name" : "User",
> "namespace" : "App",
> "fields" : [ {
> "name" : "first",
> "type" : [ "null", "string" ],
> "default" : null
> }, {
> "name" : "last",
> "type" : [ "null", "string" ],
> "default" : null
> } ]
> }
> {noformat}
>
> As you can see, there's no default values for fields. Would you please tell
> me how to obtain such values?
> Thank you.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2723) Avro Java: Obtaining default values for POJO objects with ReflectData
[
https://issues.apache.org/jira/browse/AVRO-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048534#comment-17048534
]
Andy Le commented on AVRO-2723:
---
[~rskraba] Hi. Would you please tell me how to resolve this issue? Any
recommendation? I'm thinking of creating a PR to address this issue. But I
think we need a proper discussion first.
> Avro Java: Obtaining default values for POJO objects with ReflectData
> -
>
> Key: AVRO-2723
> URL: https://issues.apache.org/jira/browse/AVRO-2723
> Project: Apache Avro
> Issue Type: New Feature
> Components: java
>Affects Versions: 1.9.1
>Reporter: Andy Le
>Priority: Critical
>
> Hi guys,
>
> I've got a simple app using Avro Reflection:
>
> {code:java}
> public class App {
> public static void main(String[] args) {
> testReflection();
> }
> static class User {
> public String first = "Andy";
> public String last = "Le";
> }
> static void testReflection(){
> // get the reflected schema for packets
> Schema schema = ReflectData.AllowNull.get().getSchema(User.class);
> System.out.println(schema.toString(true));
> }
> {code}
> The output on console will be:
> {noformat}
> {
> "type" : "record",
> "name" : "User",
> "namespace" : "App",
> "fields" : [ {
> "name" : "first",
> "type" : [ "null", "string" ],
> "default" : null
> }, {
> "name" : "last",
> "type" : [ "null", "string" ],
> "default" : null
> } ]
> }
> {noformat}
>
> As you can see, there's no default values for fields. Would you please tell
> me how to obtain such values?
> Thank you.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
