kbendick commented on issue #3044: URL: https://github.com/apache/iceberg/issues/3044#issuecomment-907856714
> > it seems like the GlueCatalog could _possibly_ just pass in `null` for the `hadoopConf` > > I tried that, but I got serialization error of the jobgraph. > The workaround was to create an empty hadoop configuration object with the constructor which has a boolean false, telling them to not load any config file (since I don't have), thus the conf object will have an empty map. > > Since is there this unnecessary hadoop conf dependency, the size of my fat jar doubled it's size to somewhere currently at 310MB. Oh my apologies. I meant that with some reconfiguration we could potentially just pass in `null`. I'm going to take a look at @jackye1995's PR though as he's the author of the original GlueCatalog and there's possibly a need to have a `HadoopConfigurableGlueCatalog` (oftentimes for example, with Spark, the hadoop configuration is used for various purposes). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
