[
https://issues.apache.org/jira/browse/PIG-32?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12551166
]
Antonio Magnaghi commented on PIG-32:
-------------------------------------
Recording in the bug log some discussion points from a conversation with
Utkarsh on the separation of front-end from back-end:
Data Storage Portion:
- add a "isContainer" method to the DataStorageElement interface
Execution Engine Portion:
- Introduce the concept of Job in the type hierarchy. This would be the result
of executing/submitting a physical plan to the execution engine
- as part of the Local Back-End re-write:
- currently co-group uses in mem data structures. we should switch over to
spillable data structures instead
- a general suggestion is to favor clarity of the code. A possibility we
discussed was that some operators like LORead, LOSplits, and intermediate
results can be removed.
> Abstraction Layer to decouple Pig from Back-End
> -----------------------------------------------
>
> Key: PIG-32
> URL: https://issues.apache.org/jira/browse/PIG-32
> Project: Pig
> Issue Type: New Feature
> Components: impl
> Reporter: Antonio Magnaghi
> Assignee: Antonio Magnaghi
> Attachments: DataStorage.diff, DataStorage20071212.diff
>
>
> I'm opening a new issue to track the development work to support an
> abstraction layer for Pig as defined at
> http://wiki.apache.org/pig/PigAbstractionLayer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
