[jira] [Commented] (AVRO-1954) Schema.Field.defaultVal() generates: Unknown datum type org.apache.avro.JsonProperties$Null

2020-03-07 Thread Sergey Shcherbakov (Jira) 


[ 
https://issues.apache.org/jira/browse/AVRO-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054158#comment-17054158
 ] 

Sergey Shcherbakov commented on AVRO-1954:
--

[~mmeziane] thank you for the workaround!

> Schema.Field.defaultVal() generates: Unknown datum type 
> org.apache.avro.JsonProperties$Null
> ---
>
> Key: AVRO-1954
> URL: https://issues.apache.org/jira/browse/AVRO-1954
> Project: Apache Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.1, 1.9.0
>Reporter: rui miranda
>Assignee: Nándor Kollár
>Priority: Minor
> Fix For: 1.9.0
>
> Attachments: unitTestDefaultNull.patch
>
>
> I was creating GenericRecords and populating some fields -- which i could not 
> find the content on some json files -- with the Schema.Field.defaultVal(). 
> It seems if the schema has explicitly set the default value to be null, the 
> records generated this way can't be written. In this case, if default value 
> is null in the schema, an instance of 
> org.apache.avro.JsonProperties.NULL_VALUE is returned by 
> Schema.Field.defaultVal().
> I created an unit test which replicates the bug. I was thinking modify the 
> class org.apache.avro.generic.GenericData to evaluate 
> org.apache.avro.JsonProperties.NULL_VALUE as null. Is this the way to go? or 
> org.apache.avro.JsonProperties.NULL_VALUE is intend for other purposes?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AVRO-1954) Schema.Field.defaultVal() generates: Unknown datum type org.apache.avro.JsonProperties$Null

2019-06-19 Thread Meziane Mehdi (JIRA) 


[ 
https://issues.apache.org/jira/browse/AVRO-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16867965#comment-16867965
 ] 

Meziane Mehdi commented on AVRO-1954:
-

Hi,

Thanks for the fix.
Is there a workaround for this bug (Avro 1.8.1)?

 

> Schema.Field.defaultVal() generates: Unknown datum type 
> org.apache.avro.JsonProperties$Null
> ---
>
> Key: AVRO-1954
> URL: https://issues.apache.org/jira/browse/AVRO-1954
> Project: Apache Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.1, 1.9.0
>Reporter: rui miranda
>Assignee: Nandor Kollar
>Priority: Minor
> Fix For: 1.9.0
>
> Attachments: unitTestDefaultNull.patch
>
>
> I was creating GenericRecords and populating some fields -- which i could not 
> find the content on some json files -- with the Schema.Field.defaultVal(). 
> It seems if the schema has explicitly set the default value to be null, the 
> records generated this way can't be written. In this case, if default value 
> is null in the schema, an instance of 
> org.apache.avro.JsonProperties.NULL_VALUE is returned by 
> Schema.Field.defaultVal().
> I created an unit test which replicates the bug. I was thinking modify the 
> class org.apache.avro.generic.GenericData to evaluate 
> org.apache.avro.JsonProperties.NULL_VALUE as null. Is this the way to go? or 
> org.apache.avro.JsonProperties.NULL_VALUE is intend for other purposes?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AVRO-1954) Schema.Field.defaultVal() generates: Unknown datum type org.apache.avro.JsonProperties$Null

2018-12-11 Thread ASF subversion and git services (JIRA) 


[ 
https://issues.apache.org/jira/browse/AVRO-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16717790#comment-16717790
 ] 

ASF subversion and git services commented on AVRO-1954:
---

Commit 10792f777f5a31f8c1e5d9a6f6c6d4ba2727ab58 in avro's branch 
refs/heads/branch-1.8 from [~nkollar]
[ https://gitbox.apache.org/repos/asf?p=avro.git;h=10792f7 ]

AVRO-1954 - Schema.Field.defaultVal() generates: Unknown datum type 
org.apache.avro.JsonProperties$Null. Contributed by Nandor Kollar

(cherry picked from commit d9338a4cf008b445ea3efbe2523288d07162ec71)


> Schema.Field.defaultVal() generates: Unknown datum type 
> org.apache.avro.JsonProperties$Null
> ---
>
> Key: AVRO-1954
> URL: https://issues.apache.org/jira/browse/AVRO-1954
> Project: Apache Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.1, 1.9.0
>Reporter: rui miranda
>Assignee: Nandor Kollar
>Priority: Minor
> Fix For: 1.9.0
>
> Attachments: unitTestDefaultNull.patch
>
>
> I was creating GenericRecords and populating some fields -- which i could not 
> find the content on some json files -- with the Schema.Field.defaultVal(). 
> It seems if the schema has explicitly set the default value to be null, the 
> records generated this way can't be written. In this case, if default value 
> is null in the schema, an instance of 
> org.apache.avro.JsonProperties.NULL_VALUE is returned by 
> Schema.Field.defaultVal().
> I created an unit test which replicates the bug. I was thinking modify the 
> class org.apache.avro.generic.GenericData to evaluate 
> org.apache.avro.JsonProperties.NULL_VALUE as null. Is this the way to go? or 
> org.apache.avro.JsonProperties.NULL_VALUE is intend for other purposes?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AVRO-1954) Schema.Field.defaultVal() generates: Unknown datum type org.apache.avro.JsonProperties$Null

2018-03-07 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16390034#comment-16390034
 ] 

ASF GitHub Bot commented on AVRO-1954:
--

derekstraka closed pull request #290: AVRO-1954 - Schema.Field.defaultVal() 
generates: Unknown datum type org.apache.avro.JsonProperties$Null
URL: https://github.com/apache/avro/pull/290
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/CHANGES.txt b/CHANGES.txt
index 7156f730e..83f214592 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -40,6 +40,9 @@ Trunk (not yet released)
 AVRO-2122: Cannot validate schemas with recursive definitions
 (Bart via Nandor Kollar)
 
+AVRO-1954: Java: Schema.Field.defaultVal() generates: Unknown datum type
+(Nandor Kollar via tomwhite)
+
 Avro 1.8.2 (10 April 2017)
 
   INCOMPATIBLE CHANGES
diff --git 
a/lang/java/avro/src/main/java/org/apache/avro/generic/GenericData.java 
b/lang/java/avro/src/main/java/org/apache/avro/generic/GenericData.java
index 67d47d99c..c7931c736 100644
--- a/lang/java/avro/src/main/java/org/apache/avro/generic/GenericData.java
+++ b/lang/java/avro/src/main/java/org/apache/avro/generic/GenericData.java
@@ -36,6 +36,7 @@
 import org.apache.avro.AvroTypeException;
 import org.apache.avro.Conversion;
 import org.apache.avro.Conversions;
+import org.apache.avro.JsonProperties;
 import org.apache.avro.LogicalType;
 import org.apache.avro.Schema;
 import org.apache.avro.Schema.Field;
@@ -744,7 +745,7 @@ public int resolveUnion(Schema union, Object datum) {
   /** Return the schema full name for a datum.  Called by {@link
* #resolveUnion(Schema,Object)}. */
   protected String getSchemaName(Object datum) {
-if (datum == null)
+if (datum == null || datum == JsonProperties.NULL_VALUE)
   return Type.NULL.getName();
 if (isRecord(datum))
   return getRecordSchema(datum).getFullName();
diff --git 
a/lang/java/avro/src/test/java/org/apache/avro/generic/TestGenericDatumWriter.java
 
b/lang/java/avro/src/test/java/org/apache/avro/generic/TestGenericDatumWriter.java
index f93a326c4..8f99cdfe4 100644
--- 
a/lang/java/avro/src/test/java/org/apache/avro/generic/TestGenericDatumWriter.java
+++ 
b/lang/java/avro/src/test/java/org/apache/avro/generic/TestGenericDatumWriter.java
@@ -24,7 +24,6 @@
 import java.io.ByteArrayInputStream;
 import java.io.ByteArrayOutputStream;
 import java.io.IOException;
-import java.io.OutputStream;
 import java.nio.ByteBuffer;
 import java.util.ConcurrentModificationException;
 import java.util.HashMap;
@@ -37,11 +36,10 @@
 import java.util.concurrent.Future;
 
 import org.apache.avro.Schema;
+import org.apache.avro.io.BinaryEncoder;
 import org.apache.avro.io.DecoderFactory;
-import org.apache.avro.io.DirectBinaryEncoder;
 import org.apache.avro.io.Encoder;
 import org.apache.avro.io.EncoderFactory;
-import org.apache.avro.io.JsonDecoder;
 import org.apache.avro.AvroTypeException;
 import org.junit.Test;
 import org.apache.avro.util.Utf8;
@@ -257,4 +255,43 @@ public void writeDoesNotAllowJavaEnumForGenericEnum() 
throws IOException {
 
 writer.write(record, encoder);
   }
+
+  @Test
+  public void writeFieldWithDefaultWithExplicitNullDefaultInSchema() throws 
Exception {
+Schema schema = schemaWithExplicitNullDefault();
+GenericRecord record = createRecordWithDefaultField(schema);
+writeObject(schema, record);
+  }
+
+  @Test
+  public void writeFieldWithDefaultWithoutExplicitNullDefaultInSchema() throws 
Exception {
+Schema schema = schemaWithoutExplicitNullDefault();
+GenericRecord record = createRecordWithDefaultField(schema);
+writeObject(schema, record);
+  }
+
+  private Schema schemaWithExplicitNullDefault() {
+String schema = 
"{\"type\":\"record\",\"name\":\"my_record\",\"namespace\":\"mytest.namespace\",\"doc\":\"doc\","
 +
+
"\"fields\":[{\"name\":\"f\",\"type\":[\"null\",\"string\"],\"doc\":\"field doc 
doc\", " +
+"\"default\":null}]}";
+return new Schema.Parser().parse(schema);
+  }
+
+  private Schema schemaWithoutExplicitNullDefault() {
+String schema = 
"{\"type\":\"record\",\"name\":\"my_record\",\"namespace\":\"mytest.namespace\",\"doc\":\"doc\","
 +
+
"\"fields\":[{\"name\":\"f\",\"type\":[\"null\",\"string\"],\"doc\":\"field doc 
doc\"}]}";
+return new Schema.Parser().parse(schema);
+  }
+
+  private void writeObject(Schema schema, GenericRecord datum) throws 
Exception {
+BinaryEncoder encoder = EncoderFactory.get().binaryEncoder(new 
ByteArrayOutputStream(), null);
+GenericDatumWriter writer = new 
GenericDatumWriter(schema);
+writer.write(schema, datum, encoder);
+  }
+
+  private GenericRecord createReco

[jira] [Commented] (AVRO-1954) Schema.Field.defaultVal() generates: Unknown datum type org.apache.avro.JsonProperties$Null

2018-03-07 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16389830#comment-16389830
 ] 

ASF GitHub Bot commented on AVRO-1954:
--

nandorKollar commented on issue #290: AVRO-1954 - Schema.Field.defaultVal() 
generates: Unknown datum type org.apache.avro.JsonProperties$Null
URL: https://github.com/apache/avro/pull/290#issuecomment-371214137
 
 
   @derekstraka I cherry-picket and 
[committed](https://github.com/apache/avro/commit/10792f777f5a31f8c1e5d9a6f6c6d4ba2727ab58)
 the fix to branch-1.8 and will be release with the next release from 1.8 
branch. If it looks good to you, you can close this PR.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Schema.Field.defaultVal() generates: Unknown datum type 
> org.apache.avro.JsonProperties$Null
> ---
>
> Key: AVRO-1954
> URL: https://issues.apache.org/jira/browse/AVRO-1954
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.1, 1.9.0
>Reporter: rui miranda
>Assignee: Nandor Kollar
>Priority: Minor
> Fix For: 1.9.0
>
> Attachments: unitTestDefaultNull.patch
>
>
> I was creating GenericRecords and populating some fields -- which i could not 
> find the content on some json files -- with the Schema.Field.defaultVal(). 
> It seems if the schema has explicitly set the default value to be null, the 
> records generated this way can't be written. In this case, if default value 
> is null in the schema, an instance of 
> org.apache.avro.JsonProperties.NULL_VALUE is returned by 
> Schema.Field.defaultVal().
> I created an unit test which replicates the bug. I was thinking modify the 
> class org.apache.avro.generic.GenericData to evaluate 
> org.apache.avro.JsonProperties.NULL_VALUE as null. Is this the way to go? or 
> org.apache.avro.JsonProperties.NULL_VALUE is intend for other purposes?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AVRO-1954) Schema.Field.defaultVal() generates: Unknown datum type org.apache.avro.JsonProperties$Null

2018-03-07 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16389823#comment-16389823
 ] 

ASF subversion and git services commented on AVRO-1954:
---

Commit 10792f777f5a31f8c1e5d9a6f6c6d4ba2727ab58 in avro's branch 
refs/heads/branch-1.8 from [~nkollar]
[ https://gitbox.apache.org/repos/asf?p=avro.git;h=10792f7 ]

AVRO-1954 - Schema.Field.defaultVal() generates: Unknown datum type 
org.apache.avro.JsonProperties$Null. Contributed by Nandor Kollar

(cherry picked from commit d9338a4cf008b445ea3efbe2523288d07162ec71)


> Schema.Field.defaultVal() generates: Unknown datum type 
> org.apache.avro.JsonProperties$Null
> ---
>
> Key: AVRO-1954
> URL: https://issues.apache.org/jira/browse/AVRO-1954
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.1, 1.9.0
>Reporter: rui miranda
>Assignee: Nandor Kollar
>Priority: Minor
> Fix For: 1.9.0
>
> Attachments: unitTestDefaultNull.patch
>
>
> I was creating GenericRecords and populating some fields -- which i could not 
> find the content on some json files -- with the Schema.Field.defaultVal(). 
> It seems if the schema has explicitly set the default value to be null, the 
> records generated this way can't be written. In this case, if default value 
> is null in the schema, an instance of 
> org.apache.avro.JsonProperties.NULL_VALUE is returned by 
> Schema.Field.defaultVal().
> I created an unit test which replicates the bug. I was thinking modify the 
> class org.apache.avro.generic.GenericData to evaluate 
> org.apache.avro.JsonProperties.NULL_VALUE as null. Is this the way to go? or 
> org.apache.avro.JsonProperties.NULL_VALUE is intend for other purposes?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AVRO-1954) Schema.Field.defaultVal() generates: Unknown datum type org.apache.avro.JsonProperties$Null

2018-03-01 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16381931#comment-16381931
 ] 

ASF GitHub Bot commented on AVRO-1954:
--

derekstraka commented on issue #290: AVRO-1954 - Schema.Field.defaultVal() 
generates: Unknown datum type org.apache.avro.JsonProperties$Null
URL: https://github.com/apache/avro/pull/290#issuecomment-369579755
 
 
   I've run into this issue a few times when using the 1.8 branch, so I 
backported the fix from #180 .  If the 1.8 branch is effectively closed, this 
PR can be rejected.  


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Schema.Field.defaultVal() generates: Unknown datum type 
> org.apache.avro.JsonProperties$Null
> ---
>
> Key: AVRO-1954
> URL: https://issues.apache.org/jira/browse/AVRO-1954
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.1, 1.9.0
>Reporter: rui miranda
>Assignee: Nandor Kollar
>Priority: Minor
> Fix For: 1.9.0
>
> Attachments: unitTestDefaultNull.patch
>
>
> I was creating GenericRecords and populating some fields -- which i could not 
> find the content on some json files -- with the Schema.Field.defaultVal(). 
> It seems if the schema has explicitly set the default value to be null, the 
> records generated this way can't be written. In this case, if default value 
> is null in the schema, an instance of 
> org.apache.avro.JsonProperties.NULL_VALUE is returned by 
> Schema.Field.defaultVal().
> I created an unit test which replicates the bug. I was thinking modify the 
> class org.apache.avro.generic.GenericData to evaluate 
> org.apache.avro.JsonProperties.NULL_VALUE as null. Is this the way to go? or 
> org.apache.avro.JsonProperties.NULL_VALUE is intend for other purposes?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AVRO-1954) Schema.Field.defaultVal() generates: Unknown datum type org.apache.avro.JsonProperties$Null

2018-03-01 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16381926#comment-16381926
 ] 

ASF GitHub Bot commented on AVRO-1954:
--

derekstraka opened a new pull request #290: AVRO-1954 - 
Schema.Field.defaultVal() generates: Unknown datum type 
org.apache.avro.JsonProperties$Null
URL: https://github.com/apache/avro/pull/290
 
 
   Backport the fix for AVRO-1954 to the 1.8 branch


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Schema.Field.defaultVal() generates: Unknown datum type 
> org.apache.avro.JsonProperties$Null
> ---
>
> Key: AVRO-1954
> URL: https://issues.apache.org/jira/browse/AVRO-1954
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.1, 1.9.0
>Reporter: rui miranda
>Assignee: Nandor Kollar
>Priority: Minor
> Fix For: 1.9.0
>
> Attachments: unitTestDefaultNull.patch
>
>
> I was creating GenericRecords and populating some fields -- which i could not 
> find the content on some json files -- with the Schema.Field.defaultVal(). 
> It seems if the schema has explicitly set the default value to be null, the 
> records generated this way can't be written. In this case, if default value 
> is null in the schema, an instance of 
> org.apache.avro.JsonProperties.NULL_VALUE is returned by 
> Schema.Field.defaultVal().
> I created an unit test which replicates the bug. I was thinking modify the 
> class org.apache.avro.generic.GenericData to evaluate 
> org.apache.avro.JsonProperties.NULL_VALUE as null. Is this the way to go? or 
> org.apache.avro.JsonProperties.NULL_VALUE is intend for other purposes?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AVRO-1954) Schema.Field.defaultVal() generates: Unknown datum type org.apache.avro.JsonProperties$Null

2018-02-14 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363834#comment-16363834
 ] 

ASF GitHub Bot commented on AVRO-1954:
--

nandorKollar closed pull request #180: AVRO-1954 - Schema.Field.defaultVal() 
generates: Unknown datum type org.apache.avro.JsonProperties$Null
URL: https://github.com/apache/avro/pull/180
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/lang/java/avro/src/main/java/org/apache/avro/generic/GenericData.java 
b/lang/java/avro/src/main/java/org/apache/avro/generic/GenericData.java
index ff0893c07..96f5ad50b 100644
--- a/lang/java/avro/src/main/java/org/apache/avro/generic/GenericData.java
+++ b/lang/java/avro/src/main/java/org/apache/avro/generic/GenericData.java
@@ -35,6 +35,7 @@
 import org.apache.avro.AvroTypeException;
 import org.apache.avro.Conversion;
 import org.apache.avro.Conversions;
+import org.apache.avro.JsonProperties;
 import org.apache.avro.LogicalType;
 import org.apache.avro.Schema;
 import org.apache.avro.Schema.Field;
@@ -743,7 +744,7 @@ public int resolveUnion(Schema union, Object datum) {
   /** Return the schema full name for a datum.  Called by {@link
* #resolveUnion(Schema,Object)}. */
   protected String getSchemaName(Object datum) {
-if (datum == null)
+if (datum == null || datum == JsonProperties.NULL_VALUE)
   return Type.NULL.getName();
 if (isRecord(datum))
   return getRecordSchema(datum).getFullName();
diff --git 
a/lang/java/avro/src/test/java/org/apache/avro/generic/TestGenericDatumWriter.java
 
b/lang/java/avro/src/test/java/org/apache/avro/generic/TestGenericDatumWriter.java
index f93a326c4..8f99cdfe4 100644
--- 
a/lang/java/avro/src/test/java/org/apache/avro/generic/TestGenericDatumWriter.java
+++ 
b/lang/java/avro/src/test/java/org/apache/avro/generic/TestGenericDatumWriter.java
@@ -24,7 +24,6 @@
 import java.io.ByteArrayInputStream;
 import java.io.ByteArrayOutputStream;
 import java.io.IOException;
-import java.io.OutputStream;
 import java.nio.ByteBuffer;
 import java.util.ConcurrentModificationException;
 import java.util.HashMap;
@@ -37,11 +36,10 @@
 import java.util.concurrent.Future;
 
 import org.apache.avro.Schema;
+import org.apache.avro.io.BinaryEncoder;
 import org.apache.avro.io.DecoderFactory;
-import org.apache.avro.io.DirectBinaryEncoder;
 import org.apache.avro.io.Encoder;
 import org.apache.avro.io.EncoderFactory;
-import org.apache.avro.io.JsonDecoder;
 import org.apache.avro.AvroTypeException;
 import org.junit.Test;
 import org.apache.avro.util.Utf8;
@@ -257,4 +255,43 @@ public void writeDoesNotAllowJavaEnumForGenericEnum() 
throws IOException {
 
 writer.write(record, encoder);
   }
+
+  @Test
+  public void writeFieldWithDefaultWithExplicitNullDefaultInSchema() throws 
Exception {
+Schema schema = schemaWithExplicitNullDefault();
+GenericRecord record = createRecordWithDefaultField(schema);
+writeObject(schema, record);
+  }
+
+  @Test
+  public void writeFieldWithDefaultWithoutExplicitNullDefaultInSchema() throws 
Exception {
+Schema schema = schemaWithoutExplicitNullDefault();
+GenericRecord record = createRecordWithDefaultField(schema);
+writeObject(schema, record);
+  }
+
+  private Schema schemaWithExplicitNullDefault() {
+String schema = 
"{\"type\":\"record\",\"name\":\"my_record\",\"namespace\":\"mytest.namespace\",\"doc\":\"doc\","
 +
+
"\"fields\":[{\"name\":\"f\",\"type\":[\"null\",\"string\"],\"doc\":\"field doc 
doc\", " +
+"\"default\":null}]}";
+return new Schema.Parser().parse(schema);
+  }
+
+  private Schema schemaWithoutExplicitNullDefault() {
+String schema = 
"{\"type\":\"record\",\"name\":\"my_record\",\"namespace\":\"mytest.namespace\",\"doc\":\"doc\","
 +
+
"\"fields\":[{\"name\":\"f\",\"type\":[\"null\",\"string\"],\"doc\":\"field doc 
doc\"}]}";
+return new Schema.Parser().parse(schema);
+  }
+
+  private void writeObject(Schema schema, GenericRecord datum) throws 
Exception {
+BinaryEncoder encoder = EncoderFactory.get().binaryEncoder(new 
ByteArrayOutputStream(), null);
+GenericDatumWriter writer = new 
GenericDatumWriter(schema);
+writer.write(schema, datum, encoder);
+  }
+
+  private GenericRecord createRecordWithDefaultField(Schema schema) {
+GenericRecord record = new GenericData.Record(schema);
+record.put("f", schema.getField("f").defaultVal());
+return record;
+  }
 }


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For qu

[jira] [Commented] (AVRO-1954) Schema.Field.defaultVal() generates: Unknown datum type org.apache.avro.JsonProperties$Null

2018-02-14 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363833#comment-16363833
 ] 

ASF GitHub Bot commented on AVRO-1954:
--

nandorKollar commented on issue #180: AVRO-1954 - Schema.Field.defaultVal() 
generates: Unknown datum type org.apache.avro.JsonProperties$Null
URL: https://github.com/apache/avro/pull/180#issuecomment-365574989
 
 
   Great! However I think this should be backported to branch-1.8 too.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Schema.Field.defaultVal() generates: Unknown datum type 
> org.apache.avro.JsonProperties$Null
> ---
>
> Key: AVRO-1954
> URL: https://issues.apache.org/jira/browse/AVRO-1954
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.1, 1.9.0
>Reporter: rui miranda
>Assignee: Nandor Kollar
>Priority: Minor
> Fix For: 1.9.0
>
> Attachments: unitTestDefaultNull.patch
>
>
> I was creating GenericRecords and populating some fields -- which i could not 
> find the content on some json files -- with the Schema.Field.defaultVal(). 
> It seems if the schema has explicitly set the default value to be null, the 
> records generated this way can't be written. In this case, if default value 
> is null in the schema, an instance of 
> org.apache.avro.JsonProperties.NULL_VALUE is returned by 
> Schema.Field.defaultVal().
> I created an unit test which replicates the bug. I was thinking modify the 
> class org.apache.avro.generic.GenericData to evaluate 
> org.apache.avro.JsonProperties.NULL_VALUE as null. Is this the way to go? or 
> org.apache.avro.JsonProperties.NULL_VALUE is intend for other purposes?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AVRO-1954) Schema.Field.defaultVal() generates: Unknown datum type org.apache.avro.JsonProperties$Null

2018-02-14 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363821#comment-16363821
 ] 

ASF GitHub Bot commented on AVRO-1954:
--

derekstraka commented on issue #180: AVRO-1954 - Schema.Field.defaultVal() 
generates: Unknown datum type org.apache.avro.JsonProperties$Null
URL: https://github.com/apache/avro/pull/180#issuecomment-365573082
 
 
   Ah, awesome.  I was using the 1.8.0 release, and I had applied this patch to 
that tag locally and it resolved my issue.  I didn't think to check in master 
to see if it was already merged.  Thanks for your help!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Schema.Field.defaultVal() generates: Unknown datum type 
> org.apache.avro.JsonProperties$Null
> ---
>
> Key: AVRO-1954
> URL: https://issues.apache.org/jira/browse/AVRO-1954
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.1, 1.9.0
>Reporter: rui miranda
>Assignee: Nandor Kollar
>Priority: Minor
> Fix For: 1.9.0
>
> Attachments: unitTestDefaultNull.patch
>
>
> I was creating GenericRecords and populating some fields -- which i could not 
> find the content on some json files -- with the Schema.Field.defaultVal(). 
> It seems if the schema has explicitly set the default value to be null, the 
> records generated this way can't be written. In this case, if default value 
> is null in the schema, an instance of 
> org.apache.avro.JsonProperties.NULL_VALUE is returned by 
> Schema.Field.defaultVal().
> I created an unit test which replicates the bug. I was thinking modify the 
> class org.apache.avro.generic.GenericData to evaluate 
> org.apache.avro.JsonProperties.NULL_VALUE as null. Is this the way to go? or 
> org.apache.avro.JsonProperties.NULL_VALUE is intend for other purposes?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AVRO-1954) Schema.Field.defaultVal() generates: Unknown datum type org.apache.avro.JsonProperties$Null

2018-02-14 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363638#comment-16363638
 ] 

ASF GitHub Bot commented on AVRO-1954:
--

nandorKollar commented on issue #180: AVRO-1954 - Schema.Field.defaultVal() 
generates: Unknown datum type org.apache.avro.JsonProperties$Null
URL: https://github.com/apache/avro/pull/180#issuecomment-365531098
 
 
   @derekstraka I've no problem with using my work as base, but I'm afraid this 
pull request is already merged to master: 
https://github.com/apache/avro/blob/master/lang/java/avro/src/main/java/org/apache/avro/generic/GenericData.java#L730
   
   I'm afraid it was left open here accidentally. Are you sure that you ran 
into the same issue?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Schema.Field.defaultVal() generates: Unknown datum type 
> org.apache.avro.JsonProperties$Null
> ---
>
> Key: AVRO-1954
> URL: https://issues.apache.org/jira/browse/AVRO-1954
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.1, 1.9.0
>Reporter: rui miranda
>Assignee: Nandor Kollar
>Priority: Minor
> Fix For: 1.9.0
>
> Attachments: unitTestDefaultNull.patch
>
>
> I was creating GenericRecords and populating some fields -- which i could not 
> find the content on some json files -- with the Schema.Field.defaultVal(). 
> It seems if the schema has explicitly set the default value to be null, the 
> records generated this way can't be written. In this case, if default value 
> is null in the schema, an instance of 
> org.apache.avro.JsonProperties.NULL_VALUE is returned by 
> Schema.Field.defaultVal().
> I created an unit test which replicates the bug. I was thinking modify the 
> class org.apache.avro.generic.GenericData to evaluate 
> org.apache.avro.JsonProperties.NULL_VALUE as null. Is this the way to go? or 
> org.apache.avro.JsonProperties.NULL_VALUE is intend for other purposes?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AVRO-1954) Schema.Field.defaultVal() generates: Unknown datum type org.apache.avro.JsonProperties$Null

2018-02-13 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363305#comment-16363305
 ] 

ASF GitHub Bot commented on AVRO-1954:
--

derekstraka commented on issue #180: AVRO-1954 - Schema.Field.defaultVal() 
generates: Unknown datum type org.apache.avro.JsonProperties$Null
URL: https://github.com/apache/avro/pull/180#issuecomment-365460261
 
 
   @nandorKollar - I ran into this issue today, and I'm interest in carrying 
this into master.  Any issues using your work as a starting point and adding 
you as signed off by on the commits?  I can run them by you first if you'd 
like.  Thanks


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Schema.Field.defaultVal() generates: Unknown datum type 
> org.apache.avro.JsonProperties$Null
> ---
>
> Key: AVRO-1954
> URL: https://issues.apache.org/jira/browse/AVRO-1954
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.1, 1.9.0
>Reporter: rui miranda
>Assignee: Nandor Kollar
>Priority: Minor
> Fix For: 1.9.0
>
> Attachments: unitTestDefaultNull.patch
>
>
> I was creating GenericRecords and populating some fields -- which i could not 
> find the content on some json files -- with the Schema.Field.defaultVal(). 
> It seems if the schema has explicitly set the default value to be null, the 
> records generated this way can't be written. In this case, if default value 
> is null in the schema, an instance of 
> org.apache.avro.JsonProperties.NULL_VALUE is returned by 
> Schema.Field.defaultVal().
> I created an unit test which replicates the bug. I was thinking modify the 
> class org.apache.avro.generic.GenericData to evaluate 
> org.apache.avro.JsonProperties.NULL_VALUE as null. Is this the way to go? or 
> org.apache.avro.JsonProperties.NULL_VALUE is intend for other purposes?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AVRO-1954) Schema.Field.defaultVal() generates: Unknown datum type org.apache.avro.JsonProperties$Null

2017-01-20 Thread Nandor Kollar (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831630#comment-15831630
 ] 

Nandor Kollar commented on AVRO-1954:
-

Thanks Tom for committing this.

> Schema.Field.defaultVal() generates: Unknown datum type 
> org.apache.avro.JsonProperties$Null
> ---
>
> Key: AVRO-1954
> URL: https://issues.apache.org/jira/browse/AVRO-1954
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.1, 1.9.0
>Reporter: rui miranda
>Assignee: Nandor Kollar
>Priority: Minor
> Fix For: 1.9.0
>
> Attachments: unitTestDefaultNull.patch
>
>
> I was creating GenericRecords and populating some fields -- which i could not 
> find the content on some json files -- with the Schema.Field.defaultVal(). 
> It seems if the schema has explicitly set the default value to be null, the 
> records generated this way can't be written. In this case, if default value 
> is null in the schema, an instance of 
> org.apache.avro.JsonProperties.NULL_VALUE is returned by 
> Schema.Field.defaultVal().
> I created an unit test which replicates the bug. I was thinking modify the 
> class org.apache.avro.generic.GenericData to evaluate 
> org.apache.avro.JsonProperties.NULL_VALUE as null. Is this the way to go? or 
> org.apache.avro.JsonProperties.NULL_VALUE is intend for other purposes?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1954) Schema.Field.defaultVal() generates: Unknown datum type org.apache.avro.JsonProperties$Null

2017-01-20 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831604#comment-15831604
 ] 

ASF subversion and git services commented on AVRO-1954:
---

Commit d9338a4cf008b445ea3efbe2523288d07162ec71 in avro's branch 
refs/heads/master from [~nkollar]
[ https://git-wip-us.apache.org/repos/asf?p=avro.git;h=d9338a4 ]

AVRO-1954 - Schema.Field.defaultVal() generates: Unknown datum type 
org.apache.avro.JsonProperties$Null. Contributed by Nandor Kollar


> Schema.Field.defaultVal() generates: Unknown datum type 
> org.apache.avro.JsonProperties$Null
> ---
>
> Key: AVRO-1954
> URL: https://issues.apache.org/jira/browse/AVRO-1954
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.1, 1.9.0
>Reporter: rui miranda
>Assignee: Nandor Kollar
>Priority: Minor
> Fix For: 1.9.0
>
> Attachments: unitTestDefaultNull.patch
>
>
> I was creating GenericRecords and populating some fields -- which i could not 
> find the content on some json files -- with the Schema.Field.defaultVal(). 
> It seems if the schema has explicitly set the default value to be null, the 
> records generated this way can't be written. In this case, if default value 
> is null in the schema, an instance of 
> org.apache.avro.JsonProperties.NULL_VALUE is returned by 
> Schema.Field.defaultVal().
> I created an unit test which replicates the bug. I was thinking modify the 
> class org.apache.avro.generic.GenericData to evaluate 
> org.apache.avro.JsonProperties.NULL_VALUE as null. Is this the way to go? or 
> org.apache.avro.JsonProperties.NULL_VALUE is intend for other purposes?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1954) Schema.Field.defaultVal() generates: Unknown datum type org.apache.avro.JsonProperties$Null

2017-01-02 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15792686#comment-15792686
 ] 

ASF GitHub Bot commented on AVRO-1954:
--

GitHub user nandorKollar opened a pull request:

https://github.com/apache/avro/pull/180

AVRO-1954 - Schema.Field.defaultVal() generates: Unknown datum type 
org.apache.avro.JsonProperties$Null

Added the test cases in the patch attached to AVRO-1954 Jira to 
TestGenericDatumWriter (since the issue is related to GenericDatumWriter), and 
modified GenericData to map JsonProperties.NULL_VALUE to Type.NULL

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nandorKollar/avro AVRO-1954

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/avro/pull/180.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #180


commit 25635e1fdd66d7275abe7e65e288f3c4ee078d97
Author: Nandor Kollar 
Date:   2017-01-02T11:00:34Z

AVRO-1954 - Schema.Field.defaultVal() generates: Unknown datum type 
org.apache.avro.JsonProperties$Null




> Schema.Field.defaultVal() generates: Unknown datum type 
> org.apache.avro.JsonProperties$Null
> ---
>
> Key: AVRO-1954
> URL: https://issues.apache.org/jira/browse/AVRO-1954
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.1, 1.9.0
>Reporter: rui miranda
>Assignee: Nandor Kollar
>Priority: Minor
> Attachments: unitTestDefaultNull.patch
>
>
> I was creating GenericRecords and populating some fields -- which i could not 
> find the content on some json files -- with the Schema.Field.defaultVal(). 
> It seems if the schema has explicitly set the default value to be null, the 
> records generated this way can't be written. In this case, if default value 
> is null in the schema, an instance of 
> org.apache.avro.JsonProperties.NULL_VALUE is returned by 
> Schema.Field.defaultVal().
> I created an unit test which replicates the bug. I was thinking modify the 
> class org.apache.avro.generic.GenericData to evaluate 
> org.apache.avro.JsonProperties.NULL_VALUE as null. Is this the way to go? or 
> org.apache.avro.JsonProperties.NULL_VALUE is intend for other purposes?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)