massdosage commented on issue #170: Add support for Iceberg MR / InputFormat and OutputFormat APIs URL: https://github.com/apache/incubator-iceberg/issues/170#issuecomment-594077199 Yes, the InputFormat in the other patch was using `org.apache.hadoop.mapreduce.InputFormat;` but in order to get this working with Hive 2.3.x we had to use `org.apache.hadoop.mapred.InputFormat`. We can get the code up onto GitHub to show and then you could pull the parts you want into Iceberg if it looks useful. We'll try get that done in a day or two. Our other attempt at it was in an iceberg-mr subproject but ended in version lock hell, it's quite possible that there are some Gradle tricks to manage versions that we're not aware of as we're not big Gradle users. The "work in progress" code for that is here: https://github.com/ExpediaGroup/incubator-iceberg/blob/hive-read-support/mr/src/main/java/org/apache/iceberg/mr/IcebergInputFormat.java. We also are aware that there are probably way better ways to get to the Iceberg metadata instead of the path/string manipulation we are currently doing but we're not entirely aware of all the ins and outs of the Iceberg APIs.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
