[
https://issues.apache.org/jira/browse/APEXMALHAR-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Saumya Mohan updated APEXMALHAR-2034:
-------------------------------------
Description:
Issue:
Avro objects are not serialized by Kryo causing the Avro GenericRecord to not
be available to downstream operators if users don't explicitly mark the stream
locality at container_local or thread_local.
Solution:
This JIRA is used to create a Module on top of AvroFileInputOperator and
AvroToPojo operators such that downstream operators will access POJO instead of
Avro GenericRecord.
In this Module, the stream between the two encapsulated operators
(AvroFileInputOperator and AvroToPojo) is set to CONTAINER_LOCAL.
This new module removes the exposure of GenericRecord to downstream operators
and instead exposes the created POJO to downstream operators.
was:
This JIRA is used to create a Module on top of AvroFileInputOperator and
AvroToPojo operator. The stream between the two operators will be set to
CONTAINER_LOCAL which is required as Avro objects are not serialized by Kryo.
This will help users to directly use the module which has the locality set to
CONTAINER_LOCAL.
> Avro File To POJO Module
> ------------------------
>
> Key: APEXMALHAR-2034
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2034
> Project: Apache Apex Malhar
> Issue Type: New Feature
> Reporter: devendra tagare
> Assignee: Saumya Mohan
>
> Issue:
> Avro objects are not serialized by Kryo causing the Avro GenericRecord to not
> be available to downstream operators if users don't explicitly mark the
> stream locality at container_local or thread_local.
> Solution:
> This JIRA is used to create a Module on top of AvroFileInputOperator and
> AvroToPojo operators such that downstream operators will access POJO instead
> of Avro GenericRecord.
> In this Module, the stream between the two encapsulated operators
> (AvroFileInputOperator and AvroToPojo) is set to CONTAINER_LOCAL.
> This new module removes the exposure of GenericRecord to downstream operators
> and instead exposes the created POJO to downstream operators.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
