[jira] [Commented] (NIFI-4893) Cannot convert Avro schemas to Record schemas with default value in arrays
[
https://issues.apache.org/jira/browse/NIFI-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16388452#comment-16388452
]
ASF GitHub Bot commented on NIFI-4893:
--
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/2480
> Cannot convert Avro schemas to Record schemas with default value in arrays
> --
>
> Key: NIFI-4893
> URL: https://issues.apache.org/jira/browse/NIFI-4893
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0
> Environment: ALL
>Reporter: Gardella Juan Pablo
>Priority: Major
> Fix For: 1.6.0
>
> Attachments: issue1.zip
>
>
> Given an Avro Schema that has a default array defined, it is not possible to
> be converted to a Nifi Record Schema.
> To reproduce the bug, try to convert the following Avro schema to Record
> Schema:
> {code}
> {
> "type": "record",
> "name": "Foo1",
> "namespace": "foo.namespace",
> "fields": [
> {
> "name": "listOfInt",
> "type": {
> "type": "array",
> "items": "int"
> },
> "doc": "array of ints",
> "default": 0
> }
> ]
> }
> {code}
>
> Using org.apache.nifi.avro.AvroTypeUtil class. Attached a maven project to
> reproduce the issue and also the fix.
> * To reproduce the bug, run "mvn clean test"
> * To test the fix, run "mvn clean test -Ppatch".
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4893) Cannot convert Avro schemas to Record schemas with default value in arrays
[
https://issues.apache.org/jira/browse/NIFI-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16388346#comment-16388346
]
ASF GitHub Bot commented on NIFI-4893:
--
Github user markap14 commented on the issue:
https://github.com/apache/nifi/pull/2480
@gardellajuanpablo looks good now! Thanks for bringing this up and sticking
with us till it's all in good shape! +1 merged to master.
> Cannot convert Avro schemas to Record schemas with default value in arrays
> --
>
> Key: NIFI-4893
> URL: https://issues.apache.org/jira/browse/NIFI-4893
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0
> Environment: ALL
>Reporter: Gardella Juan Pablo
>Priority: Major
> Fix For: 1.6.0
>
> Attachments: issue1.zip
>
>
> Given an Avro Schema that has a default array defined, it is not possible to
> be converted to a Nifi Record Schema.
> To reproduce the bug, try to convert the following Avro schema to Record
> Schema:
> {code}
> {
> "type": "record",
> "name": "Foo1",
> "namespace": "foo.namespace",
> "fields": [
> {
> "name": "listOfInt",
> "type": {
> "type": "array",
> "items": "int"
> },
> "doc": "array of ints",
> "default": 0
> }
> ]
> }
> {code}
>
> Using org.apache.nifi.avro.AvroTypeUtil class. Attached a maven project to
> reproduce the issue and also the fix.
> * To reproduce the bug, run "mvn clean test"
> * To test the fix, run "mvn clean test -Ppatch".
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4893) Cannot convert Avro schemas to Record schemas with default value in arrays
[
https://issues.apache.org/jira/browse/NIFI-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16384282#comment-16384282
]
Gardella Juan Pablo commented on NIFI-4893:
---
[~markap14] let me know if you see any problem.
> Cannot convert Avro schemas to Record schemas with default value in arrays
> --
>
> Key: NIFI-4893
> URL: https://issues.apache.org/jira/browse/NIFI-4893
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0
> Environment: ALL
>Reporter: Gardella Juan Pablo
>Priority: Major
> Attachments: issue1.zip
>
>
> Given an Avro Schema that has a default array defined, it is not possible to
> be converted to a Nifi Record Schema.
> To reproduce the bug, try to convert the following Avro schema to Record
> Schema:
> {code}
> {
> "type": "record",
> "name": "Foo1",
> "namespace": "foo.namespace",
> "fields": [
> {
> "name": "listOfInt",
> "type": {
> "type": "array",
> "items": "int"
> },
> "doc": "array of ints",
> "default": 0
> }
> ]
> }
> {code}
>
> Using org.apache.nifi.avro.AvroTypeUtil class. Attached a maven project to
> reproduce the issue and also the fix.
> * To reproduce the bug, run "mvn clean test"
> * To test the fix, run "mvn clean test -Ppatch".
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4893) Cannot convert Avro schemas to Record schemas with default value in arrays
[
https://issues.apache.org/jira/browse/NIFI-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16382441#comment-16382441
]
Gardella Juan Pablo commented on NIFI-4893:
---
A test failed :(, I will check it tomorrow.
> Cannot convert Avro schemas to Record schemas with default value in arrays
> --
>
> Key: NIFI-4893
> URL: https://issues.apache.org/jira/browse/NIFI-4893
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0
> Environment: ALL
>Reporter: Gardella Juan Pablo
>Priority: Major
> Attachments: issue1.zip
>
>
> Given an Avro Schema that has a default array defined, it is not possible to
> be converted to a Nifi Record Schema.
> To reproduce the bug, try to convert the following Avro schema to Record
> Schema:
> {code}
> {
> "type": "record",
> "name": "Foo1",
> "namespace": "foo.namespace",
> "fields": [
> {
> "name": "listOfInt",
> "type": {
> "type": "array",
> "items": "int"
> },
> "doc": "array of ints",
> "default": 0
> }
> ]
> }
> {code}
>
> Using org.apache.nifi.avro.AvroTypeUtil class. Attached a maven project to
> reproduce the issue and also the fix.
> * To reproduce the bug, run "mvn clean test"
> * To test the fix, run "mvn clean test -Ppatch".
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4893) Cannot convert Avro schemas to Record schemas with default value in arrays
[
https://issues.apache.org/jira/browse/NIFI-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16382433#comment-16382433
]
Gardella Juan Pablo commented on NIFI-4893:
---
[~markap14] applied the suggested fix. Ready to be reviewed. Thanks for the
suggestion and your time.
> Cannot convert Avro schemas to Record schemas with default value in arrays
> --
>
> Key: NIFI-4893
> URL: https://issues.apache.org/jira/browse/NIFI-4893
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0
> Environment: ALL
>Reporter: Gardella Juan Pablo
>Priority: Major
> Attachments: issue1.zip
>
>
> Given an Avro Schema that has a default array defined, it is not possible to
> be converted to a Nifi Record Schema.
> To reproduce the bug, try to convert the following Avro schema to Record
> Schema:
> {code}
> {
> "type": "record",
> "name": "Foo1",
> "namespace": "foo.namespace",
> "fields": [
> {
> "name": "listOfInt",
> "type": {
> "type": "array",
> "items": "int"
> },
> "doc": "array of ints",
> "default": 0
> }
> ]
> }
> {code}
>
> Using org.apache.nifi.avro.AvroTypeUtil class. Attached a maven project to
> reproduce the issue and also the fix.
> * To reproduce the bug, run "mvn clean test"
> * To test the fix, run "mvn clean test -Ppatch".
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4893) Cannot convert Avro schemas to Record schemas with default value in arrays
[
https://issues.apache.org/jira/browse/NIFI-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16382057#comment-16382057
]
ASF GitHub Bot commented on NIFI-4893:
--
Github user gardellajuanpablo commented on the issue:
https://github.com/apache/nifi/pull/2480
@markap14 let me test it and I will let you know if it works in our
environment. Thanks a lot.
> Cannot convert Avro schemas to Record schemas with default value in arrays
> --
>
> Key: NIFI-4893
> URL: https://issues.apache.org/jira/browse/NIFI-4893
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0
> Environment: ALL
>Reporter: Gardella Juan Pablo
>Priority: Major
> Attachments: issue1.zip
>
>
> Given an Avro Schema that has a default array defined, it is not possible to
> be converted to a Nifi Record Schema.
> To reproduce the bug, try to convert the following Avro schema to Record
> Schema:
> {code}
> {
> "type": "record",
> "name": "Foo1",
> "namespace": "foo.namespace",
> "fields": [
> {
> "name": "listOfInt",
> "type": {
> "type": "array",
> "items": "int"
> },
> "doc": "array of ints",
> "default": 0
> }
> ]
> }
> {code}
>
> Using org.apache.nifi.avro.AvroTypeUtil class. Attached a maven project to
> reproduce the issue and also the fix.
> * To reproduce the bug, run "mvn clean test"
> * To test the fix, run "mvn clean test -Ppatch".
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4893) Cannot convert Avro schemas to Record schemas with default value in arrays
[
https://issues.apache.org/jira/browse/NIFI-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16382050#comment-16382050
]
ASF GitHub Bot commented on NIFI-4893:
--
Github user markap14 commented on the issue:
https://github.com/apache/nifi/pull/2480
@gardellajuanpablo I think for this case we should avoid making any changes
to RecordField or DataTypeUtils at all. This is an oddity of Avro specifically.
I do think that it makes sense to honor the way that Avro deals with this case,
but I think this should be done purely by addressing the issue in AvroTypeUtil.
I think we can just update the AvroTypeUtil#createSchema(Schema, String,
SchemaIdentifier) with something like this:
```
Object defaultValue = field.defaultVal();
if (fieldSchema.getType() == Schema.Type.ARRAY &&
!DataTypeUtils.isArrayTypeCompatible(defaultValue)) {
defaultValue = new Object[0];
}
recordFields.add(new RecordField(fieldName, dataType,
defaultValue, field.aliases(), nullable));
```
And I think that will address the issue without breaking anything else.
Thoughts?
> Cannot convert Avro schemas to Record schemas with default value in arrays
> --
>
> Key: NIFI-4893
> URL: https://issues.apache.org/jira/browse/NIFI-4893
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0
> Environment: ALL
>Reporter: Gardella Juan Pablo
>Priority: Major
> Attachments: issue1.zip
>
>
> Given an Avro Schema that has a default array defined, it is not possible to
> be converted to a Nifi Record Schema.
> To reproduce the bug, try to convert the following Avro schema to Record
> Schema:
> {code}
> {
> "type": "record",
> "name": "Foo1",
> "namespace": "foo.namespace",
> "fields": [
> {
> "name": "listOfInt",
> "type": {
> "type": "array",
> "items": "int"
> },
> "doc": "array of ints",
> "default": 0
> }
> ]
> }
> {code}
>
> Using org.apache.nifi.avro.AvroTypeUtil class. Attached a maven project to
> reproduce the issue and also the fix.
> * To reproduce the bug, run "mvn clean test"
> * To test the fix, run "mvn clean test -Ppatch".
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4893) Cannot convert Avro schemas to Record schemas with default value in arrays
[
https://issues.apache.org/jira/browse/NIFI-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370269#comment-16370269
]
ASF GitHub Bot commented on NIFI-4893:
--
Github user gardellajuanpablo commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2480#discussion_r169389143
--- Diff:
nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/RecordField.java
---
@@ -76,8 +77,12 @@ public RecordField(final String fieldName, final
DataType dataType, final Object
} else {
this.aliases = Collections.unmodifiableSet(aliases);
}
-
-this.defaultValue = defaultValue;
+if (RecordFieldType.ARRAY.equals(dataType.getFieldType()) &&
(defaultValue instanceof String
+|| defaultValue instanceof Number || defaultValue
instanceof Boolean)) {
--- End diff --
Yes, it solves the reported issue, but it makes the test fails. It is
required to think another solution approach. It is not as simple I initially
thought
> Cannot convert Avro schemas to Record schemas with default value in arrays
> --
>
> Key: NIFI-4893
> URL: https://issues.apache.org/jira/browse/NIFI-4893
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0
> Environment: ALL
>Reporter: Gardella Juan Pablo
>Priority: Major
> Attachments: issue1.zip
>
>
> Given an Avro Schema that has a default array defined, it is not possible to
> be converted to a Nifi Record Schema.
> To reproduce the bug, try to convert the following Avro schema to Record
> Schema:
> {code}
> {
> "type": "record",
> "name": "Foo1",
> "namespace": "foo.namespace",
> "fields": [
> {
> "name": "listOfInt",
> "type": {
> "type": "array",
> "items": "int"
> },
> "doc": "array of ints",
> "default": 0
> }
> ]
> }
> {code}
>
> Using org.apache.nifi.avro.AvroTypeUtil class. Attached a maven project to
> reproduce the issue and also the fix.
> * To reproduce the bug, run "mvn clean test"
> * To test the fix, run "mvn clean test -Ppatch".
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4893) Cannot convert Avro schemas to Record schemas with default value in arrays
[
https://issues.apache.org/jira/browse/NIFI-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16370254#comment-16370254
]
ASF GitHub Bot commented on NIFI-4893:
--
Github user zenfenan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2480#discussion_r169385890
--- Diff:
nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/RecordField.java
---
@@ -76,8 +77,12 @@ public RecordField(final String fieldName, final
DataType dataType, final Object
} else {
this.aliases = Collections.unmodifiableSet(aliases);
}
-
-this.defaultValue = defaultValue;
+if (RecordFieldType.ARRAY.equals(dataType.getFieldType()) &&
(defaultValue instanceof String
+|| defaultValue instanceof Number || defaultValue
instanceof Boolean)) {
--- End diff --
This will cause
[TestAvroReaderWithEmbeddedSchema](https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/test/java/org/apache/nifi/avro/TestAvroReaderWithEmbeddedSchema.java)
to fail.
> Cannot convert Avro schemas to Record schemas with default value in arrays
> --
>
> Key: NIFI-4893
> URL: https://issues.apache.org/jira/browse/NIFI-4893
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0
> Environment: ALL
>Reporter: Gardella Juan Pablo
>Priority: Major
> Attachments: issue1.zip
>
>
> Given an Avro Schema that has a default array defined, it is not possible to
> be converted to a Nifi Record Schema.
> To reproduce the bug, try to convert the following Avro schema to Record
> Schema:
> {code}
> {
> "type": "record",
> "name": "Foo1",
> "namespace": "foo.namespace",
> "fields": [
> {
> "name": "listOfInt",
> "type": {
> "type": "array",
> "items": "int"
> },
> "doc": "array of ints",
> "default": 0
> }
> ]
> }
> {code}
>
> Using org.apache.nifi.avro.AvroTypeUtil class. Attached a maven project to
> reproduce the issue and also the fix.
> * To reproduce the bug, run "mvn clean test"
> * To test the fix, run "mvn clean test -Ppatch".
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
