[ https://issues.apache.org/jira/browse/IMPALA-10332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Becker resolved IMPALA-10332. ------------------------------------ Resolution: Implemented > Add file formats to HdfsScanNode's thrift representation and codegen for those > ------------------------------------------------------------------------------ > > Key: IMPALA-10332 > URL: https://issues.apache.org/jira/browse/IMPALA-10332 > Project: IMPALA > Issue Type: Improvement > Components: Backend, Frontend > Reporter: Daniel Becker > Assignee: Daniel Becker > Priority: Major > > List all file formats that a HdfsScanNode needs to process in any fragment > instance. It is possible that some file formats will not be needed in all > fragment instances. > This is a step towards sharing codegen between different impala backends. > Using the file formats provided in the thrift file, a backend can codegen > code for file formats that are not needed in its own process but are needed > in other fragment instances running on other backends, and the resulting > binary can be shared between multiple backends. > Codegenning for file formats will be done based on the thrift message and not > on what is needed for the actual backend. This leads to some extra work in > case a file format is not needed for the current backend and codegen sharing > is not available (at this point it is not implemented). However, the overall > number of such cases is low. > Also adding the file formats to the node's explain string. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org