Saumya Mohan commented on APEXMALHAR-2034:

Hi Thomas, this JIRA is just the first step toward improving the Avro input 
functionality. As part of this JIRA we're creating a module to encapsulate Avro 
Container File -> Generic Record -> POJO transformation which user can use 
directly. Further enhancements will take place as part of separate JIRAs.

> Avro File Input Operator
> ------------------------
>                 Key: APEXMALHAR-2034
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2034
>             Project: Apache Apex Malhar
>          Issue Type: New Feature
>            Reporter: devendra tagare
>            Assignee: Saumya Mohan
> This operator would extend the AbstractFileInputOperator to read Avro 
> Container files.
> Input would be an Avro Container File.
> Output would be a GenericRecord.
> There would be 2 additional optional ports,
> 1.FilePort - for completed files.
> 2.FailedRecordsPort - this will capture fileName,Offset & error message as a 
> string for handling by a downstream operator.
> This operator can be used in isolation or with the AvroToPojo operator to 
> read an Avro record and convert it to a POJO.
> ---------------------------------------------------------------------------------------------------------------------
> This JIRA is used to create a Module on top of AvroFileInputOperator and 
> AvroToPojo operator. The stream between the two operators will be set to 
> CONTAINER_LOCAL which is required as Avro objects are not serialized by Kryo. 
> This will help users to directly use the module which has the locality set to 

This message was sent by Atlassian JIRA

Reply via email to