[
https://issues.apache.org/jira/browse/NIFI-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16510183#comment-16510183
]
ASF GitHub Bot commented on NIFI-5213:
--------------------------------------
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2718#discussion_r194883172
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/test/java/org/apache/nifi/avro/TestAvroReaderWithExplicitSchema.java
---
@@ -0,0 +1,62 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.avro;
+
+import org.apache.avro.Schema;
+import org.apache.nifi.serialization.SimpleRecordSchema;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.junit.Test;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+
+public class TestAvroReaderWithExplicitSchema {
+
+ @Test
+ public void testAvroExplicitReaderWithSchemalessFile() throws
Exception {
+ File avroFileWithEmbeddedSchema = new
File("src/test/resources/avro/avro_schemaless.avro");
+ FileInputStream fileInputStream = new
FileInputStream(avroFileWithEmbeddedSchema);
+ Schema dataSchema = new Schema.Parser().parse(new
File("src/test/resources/avro/avro_schemaless.avsc"));
+ RecordSchema recordSchema = new
SimpleRecordSchema(dataSchema.toString(), AvroTypeUtil.AVRO_SCHEMA_FORMAT,
null);
+
+ AvroReaderWithExplicitSchema avroReader = new
AvroReaderWithExplicitSchema(fileInputStream, recordSchema, dataSchema);
+ avroReader.nextAvroRecord();
+ }
+
+ @Test
+ public void testAvroExplicitReaderWithEmbeddedSchemaFile() throws
Exception {
+ File avroFileWithEmbeddedSchema = new
File("src/test/resources/avro/avro_embed_schema.avro");
+ FileInputStream fileInputStream = new
FileInputStream(avroFileWithEmbeddedSchema);
+ Schema dataSchema = new Schema.Parser().parse(new
File("src/test/resources/avro/avro_schemaless.avsc"));
+ RecordSchema recordSchema = new
SimpleRecordSchema(dataSchema.toString(), AvroTypeUtil.AVRO_SCHEMA_FORMAT,
null);
+
+ AvroReaderWithExplicitSchema avroReader = new
AvroReaderWithExplicitSchema(fileInputStream, recordSchema, dataSchema);
+ avroReader.nextAvroRecord();
+ }
+
+ @Test(expected = IOException.class)
--- End diff --
No from the constructor. I updated the test to make that more clear, thanks!
> Allow AvroReader with explicit schema to read files with embedded schema
> ------------------------------------------------------------------------
>
> Key: NIFI-5213
> URL: https://issues.apache.org/jira/browse/NIFI-5213
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Matt Burgess
> Assignee: Matt Burgess
> Priority: Minor
>
> AvroReader allows the choice of schema access strategy from such options as
> Use Embedded Schema, Use Schema Name, Use Schema Text, etc. If the incoming
> Avro files will have embedded schemas, then Use Embedded Schema is best
> practice for the Avro Reader. However it is not intuitive that if the same
> schema that is embedded in the file is specified by name (using a schema
> registry) or explicitly via Schema Text, that errors can occur. This has been
> noticed in QueryRecord for example, and the error is also not intuitive or
> descriptive (it is often an ArrayIndexOutOfBoundsException).
> To provide a better user experience, it would be an improvement for
> AvroReader to be able to successfully process Avro files with embedded
> schemas, even when the Schema Access Strategy is not "Use Embedded Schema".
> Of course, the explicit schema would have to match the embedded schema, or an
> error would be reported (and rightfully so).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
