[jira] [Commented] (AVRO-1927) If a default value is set, Avro allows null values in non-nullable fields.
[
https://issues.apache.org/jira/browse/AVRO-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547243#comment-16547243
]
Ryan Blue commented on AVRO-1927:
-
Sounds like the problem here is actually that there is no validation in the
avro-specific builders or in object creation. Avro is doing the right thing and
not allowing you to serialize invalid data.
I think it makes sense to add null checks to the generated builders. Feel free
to fix this and open a PR!
> If a default value is set, Avro allows null values in non-nullable fields.
> --
>
> Key: AVRO-1927
> URL: https://issues.apache.org/jira/browse/AVRO-1927
> Project: Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.8.1
>Reporter: Andreas Maier
>Priority: Major
>
> With an avro schema like
> {code}
> {
> "name": "myfield",
> "type": "string",
> "default": ""
> }
> {code}
> the following code should throw an exception
> {code}
> MyObject myObject = MyObject.newBuilder().setMyfield(null).build();
> {code}
> But instead the value of myfield is set to null, which causes an exception
> later when serializing myObject, because null is not a valid value for
> myfield.
> I believe in this case setMyfield(null) should throw an exception,
> independent of the value of default.
> See also
> https://stackoverflow.com/questions/38509279/generated-avro-builder-set-null-doesnt-overwrite-with-default
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AVRO-1927) If a default value is set, Avro allows null values in non-nullable fields.
[
https://issues.apache.org/jira/browse/AVRO-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16527320#comment-16527320
]
Mykola Polonskyi commented on AVRO-1927:
hello (bump message)
*same behavior from builder on 1.8.2 and same error onto serialization call*
can any one share reference for spec that explain this behavior if its valid?
or clarify status
tnx
> If a default value is set, Avro allows null values in non-nullable fields.
> --
>
> Key: AVRO-1927
> URL: https://issues.apache.org/jira/browse/AVRO-1927
> Project: Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.8.1
>Reporter: Andreas Maier
>Priority: Major
>
> With an avro schema like
> {code}
> {
> "name": "myfield",
> "type": "string",
> "default": ""
> }
> {code}
> the following code should throw an exception
> {code}
> MyObject myObject = MyObject.newBuilder().setMyfield(null).build();
> {code}
> But instead the value of myfield is set to null, which causes an exception
> later when serializing myObject, because null is not a valid value for
> myfield.
> I believe in this case setMyfield(null) should throw an exception,
> independent of the value of default.
> See also
> https://stackoverflow.com/questions/38509279/generated-avro-builder-set-null-doesnt-overwrite-with-default
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AVRO-1927) If a default value is set, Avro allows null values in non-nullable fields.
[
https://issues.apache.org/jira/browse/AVRO-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568879#comment-15568879
]
Andreas Maier commented on AVRO-1927:
-
So I had a look at the generated Java code. And strangely enough, this code
only throws an exception, if no default value is set:
{code}
protected void validate(Field field, Object value) {
if(!isValidValue(field, value)) {
if(field.defaultValue() == null) { // why this check?
throw new AvroRuntimeException("Field " + field + " does not
accept null values");
}
}
}
{code}
I don't understand why Avro checks, if {{if(field.defaultValue() == null)}}
before throwing an exception. In my opinion it should always throw an exception
if the field value is invalid.
> If a default value is set, Avro allows null values in non-nullable fields.
> --
>
> Key: AVRO-1927
> URL: https://issues.apache.org/jira/browse/AVRO-1927
> Project: Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.8.1
>Reporter: Andreas Maier
> Labels: newbie
>
> With an avro schema like
> {code}
> {
> "name": "myfield",
> "type": "string",
> "default": ""
> }
> {code}
> the following code should throw an exception
> {code}
> MyObject myObject = MyObject.newBuilder().setMyfield(null).build();
> {code}
> But instead the value of myfield is set to null, which causes an exception
> later when serializing myObject, because null is not a valid value for
> myfield.
> I believe in this case setMyfield(null) should throw an exception,
> independent of the value of default.
> See also
> https://stackoverflow.com/questions/38509279/generated-avro-builder-set-null-doesnt-overwrite-with-default
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (AVRO-1927) If a default value is set, Avro allows null values in non-nullable fields.
[
https://issues.apache.org/jira/browse/AVRO-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568867#comment-15568867
]
Andreas Maier commented on AVRO-1927:
-
Sorry for the long delay. I was on vacation.
I tried the code you suggested. You are right, it does throw a
NullPointerException when I try to write the object:
{code}
java.lang.NullPointerException: null of string of de.am.MyObject
at
org.apache.avro.generic.GenericDatumWriter.npe(GenericDatumWriter.java:132)
at
org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:126)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:60)
at de.am.MyObjectTest.myObjectTest2(MyObjectTest.java:27)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:119)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
at
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Caused by: java.lang.NullPointerException
at
org.apache.avro.specific.SpecificDatumWriter.writeString(SpecificDatumWriter.java:67)
at
org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:115)
at
org.apache.avro.specific.SpecificDatumWriter.writeField(SpecificDatumWriter.java:87)
at
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:143)
at
org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:105)
... 30 more
{code}
> If a default value is set, Avro allows null values in non-nullable fields.
> --
>
> Key: AVRO-1927
> URL: https://issues.apache.org/jira/browse/AVRO-1927
> Project: Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.8.1
>Reporter: Andreas Maier
> Labels: newbie
>
> With an avro schema like
> {code}
> {
> "name": "myfield",
> "type": "string",
> "default": ""
> }
> {code}
> the following code should throw an exception
> {code}
> MyObject myObject = MyObject.newBuilder().setMyfield(null).build();
> {code}
> But instead the value of myfield is set to null, which causes an exception
> later when serializing myObject, because null is not a valid value for
> myfield.
> I believe in this case setMyfield(null) should throw an exception,
> independent of the value of default.
> See also
> https://stackoverflow.com/questions/38509279/generated-avro-builder-set-null-doesnt-overwrite-with-default
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (AVRO-1927) If a default value is set, Avro allows null values in non-nullable fields.
[
https://issues.apache.org/jira/browse/AVRO-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522012#comment-15522012
]
Yibing Shi commented on AVRO-1927:
--
[~asmaier], currently the generated code doesn't check whether we are setting a
null value for a non nullable column. In your case, null is a valid Java String
value and thus it works. However, I believe AVRO would throw an exception when
you try to write your object. Have you tried that? Does it fail or succeed?
{code}
ByteArrayOutputStream os = new ByteArrayOutputStream(1024);
Encoder encoder = EncoderFactory.get().binaryEncoder(os, null);
SpecificDatumWriter writer = new
SpecificDatumWriter(MyObject.getClassSchema());
writer.write(myObject, encoder);
{code}
> If a default value is set, Avro allows null values in non-nullable fields.
> --
>
> Key: AVRO-1927
> URL: https://issues.apache.org/jira/browse/AVRO-1927
> Project: Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.8.1
>Reporter: Andreas Maier
> Labels: newbie
>
> With an avro schema like
> {code}
> {
> "name": "myfield",
> "type": "string",
> "default": ""
> }
> {code}
> the following code should throw an exception
> {code}
> MyObject myObject = MyObject.newBuilder().setMyfield(null).build();
> {code}
> But instead the value of myfield is set to null, which causes an exception
> later when serializing myObject, because null is not a valid value for
> myfield.
> I believe in this case setMyfield(null) should throw an exception,
> independent of the value of default.
> See also
> https://stackoverflow.com/questions/38509279/generated-avro-builder-set-null-doesnt-overwrite-with-default
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
