[ https://issues.apache.org/jira/browse/APEXMALHAR-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16122764#comment-16122764 ]
Thomas Weise commented on APEXMALHAR-2034: ------------------------------------------ I would recommend to review the design of this overall and make sure the Avro spec is properly supported (including logical types from 1.8). Also, Avro is a serialization framework that by itself supports multiple wire representations. I'm not sure why GenericRecord would be exposed to a downstream operator. It seems that an abstract class should provide the GenericRecord and leave it to a derived class to provide a result. > Avro File Input Operator > ------------------------ > > Key: APEXMALHAR-2034 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2034 > Project: Apache Apex Malhar > Issue Type: New Feature > Reporter: devendra tagare > Assignee: Saumya Mohan > > This operator would extend the AbstractFileInputOperator to read Avro > Container files. > Input would be an Avro Container File. > Output would be a GenericRecord. > There would be 2 additional optional ports, > 1.FilePort - for completed files. > 2.FailedRecordsPort - this will capture fileName,Offset & error message as a > string for handling by a downstream operator. > This operator can be used in isolation or with the AvroToPojo operator to > read an Avro record and convert it to a POJO. > --------------------------------------------------------------------------------------------------------------------- > This JIRA is used to create a Module on top of AvroFileInputOperator and > AvroToPojo operator. The stream between the two operators will be set to > CONTAINER_LOCAL which is required as Avro objects are not serialized by Kryo. > This will help users to directly use the module which has the locality set to > CONTAINER_LOCAL. -- This message was sent by Atlassian JIRA (v6.4.14#64029)