teabot commented on issue #170: Add support for Iceberg MR / InputFormat and OutputFormat APIs URL: https://github.com/apache/incubator-iceberg/issues/170#issuecomment-595243046 Firstly, what has been shared is very much a work in progress, it does not represent a complete solution. To your points: 1. Yes, it would be beneficial to do this. However, we are first focusing on getting records into Hive. 2. This is a good question. ORC does this already and so we'll attempt to use the same mechanism. The required information appears to be generally available in the job configuration. 3. Agreed, we should store catalogue information in the metastore. Thanks, Elliot. On Thu, 5 Mar 2020 at 12:51, Saisai Shao <[email protected]> wrote: > Sorry I still have several questions about the implementation of read > path, would like know how you think about this. > > 1. Are we going to implement as a storage handler (non-native table) > or just input/output format? From what I understanding, seems like > HiveStorageHandler is more suitable. > 2. How do we get filter expressions from Hive and map to Iceberg' > internal ones, also to push the expressions? Spark will get the filter > expressions to reader, not sure if Hive has such thing. > 3. How do we leverage Iceberg's catalog (HiveCatalog)? In the current > implementation, it only uses HadoopTables with location property, not > sure how do we leverage Iceberg's catalog implementation. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/apache/incubator-iceberg/issues/170?email_source=notifications&email_token=AABX4VTTFFUTOHZZWLZ4MHDRF6N3NA5CNFSM4HH46HTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEN5DSXQ#issuecomment-595212638>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AABX4VUV5UCA7SL5DOZXOMDRF6N3NANCNFSM4HH46HTA> > . >
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
