[
https://issues.apache.org/jira/browse/APEXMALHAR-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Saumya Mohan updated APEXMALHAR-2034:
-------------------------------------
Description:
Issue:
Avro objects are not serialized by Kryo causing the Avro GenericRecord to not
be available to downstream operators if users don't explicitly mark the stream
locality at container_local or thread_local.
Solution:
This JIRA is used to create a Module on top of AvroFileInputOperator and
AvroToPojo operators such that downstream operators will access POJO instead of
Avro GenericRecord. It, therefore, removes the exposure of GenericRecord to
downstream operators and instead exposes the created POJO to downstream
operators.
In this Module, the stream between the two encapsulated operators
(AvroFileInputOperator and AvroToPojo) is set to CONTAINER_LOCAL.
Along with this new module, existing avro support files are moved from contrib
module to a new 'avro' module.
was:
Issue:
Avro objects are not serialized by Kryo causing the Avro GenericRecord to not
be available to downstream operators if users don't explicitly mark the stream
locality at container_local or thread_local.
Solution:
This JIRA is used to create a Module on top of AvroFileInputOperator and
AvroToPojo operators such that downstream operators will access POJO instead of
Avro GenericRecord.
In this Module, the stream between the two encapsulated operators
(AvroFileInputOperator and AvroToPojo) is set to CONTAINER_LOCAL.
This new module removes the exposure of GenericRecord to downstream operators
and instead exposes the created POJO to downstream operators.
> Avro File To POJO Module
> ------------------------
>
> Key: APEXMALHAR-2034
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2034
> Project: Apache Apex Malhar
> Issue Type: New Feature
> Reporter: devendra tagare
> Assignee: Saumya Mohan
>
> Issue:
> Avro objects are not serialized by Kryo causing the Avro GenericRecord to not
> be available to downstream operators if users don't explicitly mark the
> stream locality at container_local or thread_local.
> Solution:
> This JIRA is used to create a Module on top of AvroFileInputOperator and
> AvroToPojo operators such that downstream operators will access POJO instead
> of Avro GenericRecord. It, therefore, removes the exposure of GenericRecord
> to downstream operators and instead exposes the created POJO to downstream
> operators.
> In this Module, the stream between the two encapsulated operators
> (AvroFileInputOperator and AvroToPojo) is set to CONTAINER_LOCAL.
> Along with this new module, existing avro support files are moved from
> contrib module to a new 'avro' module.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
