[
https://issues.apache.org/jira/browse/CALCITE-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16858962#comment-16858962
]
Khai Tran commented on CALCITE-3122:
------------------------------------
Thanks a lot, [~julianhyde]. This is my first time contributing to open source,
so really excited.
I have tests for all Pig operators and expressions using EMP/DEPT tables, but
those tests could not cover expressions on nested data types due to the flatten
schemas. So nested data types are tested on actual LinkedIn pig scripts, which
cannot be replicated here.
Let me create a pull request first and hope to get more feedback from the
community.
> Convert Pig Latin scripts into Calcite logical plan
> ----------------------------------------------------
>
> Key: CALCITE-3122
> URL: https://issues.apache.org/jira/browse/CALCITE-3122
> Project: Calcite
> Issue Type: New Feature
> Components: core, piglet
> Reporter: Khai Tran
> Priority: Major
>
> We create an internal Calcite repo at LinkedIn and develop APIs to parse any
> Pig Latin scripts into Calcite logical plan. The code was tested in nearly
> ~1000 Pig scripts written at LinkedIn.
> Changes:
> 1. piglet: main conversion code live there, include:
> * APIs to convert any Pig scripts into RelNode plans or SQL statements
> * Use Pig Grunt parser to parse Pig Latin scripts into Pig logical plan
> (DAGs)
> * Convert Pig schemas into RelDatatype
> * Traverse through Pig expression plan and convert Pig expressions into
> RexNodes
> * Map some basic Pig UDFs to Calcite SQL operators
> * Build Calcite UDFs for any other Pig UDFs, including UDFs written in both
> Java and Python
> * Traverse (DFS) through Pig logical plans to convert each Pig logical nodes
> to RelNodes
> * Have an optimizer rule to optimize Pig group/cogroup into Aggregate
> operators
> 2. core:
> * Implement other RelNode in Rel2Sql so that Pig can be translated into SQL
> * Other minor changes in a few other classes to make Pig to Calcite works
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
