[ https://issues.apache.org/jira/browse/HIVE-896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13568425#comment-13568425 ]
Harish Butani commented on HIVE-896: ------------------------------------ Ashutosh, So the plan looks like this: ... -> ReduceSink -> Extract -> PTFOp We don't know what columns a PTF will access; the contract is it has access to all columns in its input. So we don't want any Column Pruning to happen. So we don't put a Select Op before the Reduce Sink. At translation time we see all the Columns, including the VCs. It appears as though during optimization VCs are carried forward only if required; so at runtime the ColumnExprNodeDescs are referring to the wrong internalNames. Does this make sense? Is there a way to carry forward the VCs when a PTF is present. The other option is (which we have taken is) to say VCs are not available to PTFs. Having said this, when the PTF is Windowing, we do know the columns being referred; so we should put a Select Op in front of the ReduceSink. > Add LEAD/LAG/FIRST/LAST analytical windowing functions to Hive. > --------------------------------------------------------------- > > Key: HIVE-896 > URL: https://issues.apache.org/jira/browse/HIVE-896 > Project: Hive > Issue Type: New Feature > Components: OLAP, UDF > Reporter: Amr Awadallah > Priority: Minor > Attachments: DataStructs.pdf, HIVE-896.1.patch.txt, > Hive-896.2.patch.txt > > > Windowing functions are very useful for click stream processing and similar > time-series/sliding-window analytics. > More details at: > http://download-west.oracle.com/docs/cd/B13789_01/server.101/b10736/analysis.htm#i1006709 > http://download-west.oracle.com/docs/cd/B13789_01/server.101/b10736/analysis.htm#i1007059 > http://download-west.oracle.com/docs/cd/B13789_01/server.101/b10736/analysis.htm#i1007032 > -- amr -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira