[
https://issues.apache.org/jira/browse/APEXMALHAR-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Saumya Mohan updated APEXMALHAR-2034:
-
Description:
This operator would extend the AbstractFileInputOperator to read Avro Container
files.
Input would be an Avro Container File.
Output would be a GenericRecord.
There would be 2 additional optional ports,
1.FilePort - for completed files.
2.FailedRecordsPort - this will capture fileName,Offset & error message as a
string for handling by a downstream operator.
This operator can be used in isolation or with the AvroToPojo operator to read
an Avro record and convert it to a POJO.
-
This JIRA is used to create a Module on top of AvroFileInputOperator and
AvroToPojo operator. The stream between the two operators will be set to
CONTAINER_LOCAL which is required as Avro objects are not serialized by Kryo.
This will help users to directly use the module which has the locality set to
CONTAINER_LOCAL.
was:
This operator would extend the AbstractFileInputOperator to read Avro Container
files.
Input would be an Avro Container File.
Output would be a GenericRecord.
There would be 2 additional optional ports,
1.FilePort - for completed files.
2.FailedRecordsPort - this will capture fileName,Offset & error message as a
string for handling by a downstream operator.
This operator can be used in isolation or with the AvroToPojo operator to read
an Avro record and convert it to a POJO.
> Avro File Input Operator
>
>
> Key: APEXMALHAR-2034
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2034
> Project: Apache Apex Malhar
> Issue Type: New Feature
>Reporter: devendra tagare
>Assignee: Saumya Mohan
>
> This operator would extend the AbstractFileInputOperator to read Avro
> Container files.
> Input would be an Avro Container File.
> Output would be a GenericRecord.
> There would be 2 additional optional ports,
> 1.FilePort - for completed files.
> 2.FailedRecordsPort - this will capture fileName,Offset & error message as a
> string for handling by a downstream operator.
> This operator can be used in isolation or with the AvroToPojo operator to
> read an Avro record and convert it to a POJO.
> -
> This JIRA is used to create a Module on top of AvroFileInputOperator and
> AvroToPojo operator. The stream between the two operators will be set to
> CONTAINER_LOCAL which is required as Avro objects are not serialized by Kryo.
> This will help users to directly use the module which has the locality set to
> CONTAINER_LOCAL.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
