jackye1995 commented on a change in pull request #3752:
URL: https://github.com/apache/iceberg/pull/3752#discussion_r773355451
##########
File path:
mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java
##########
@@ -165,12 +167,47 @@ public DecomposedPredicate decomposePredicate(JobConf
jobConf, Deserializer dese
/**
* Returns the Table serialized to the configuration based on the table name.
+ * If configuration is missing from the FileIO of the table, it will be
populated with the input config.
+ *
* @param config The configuration used to get the data from
* @param name The name of the table we need as returned by
TableDesc.getTableName()
* @return The Table
*/
public static Table table(Configuration config, String name) {
- return
SerializationUtil.deserializeFromBase64(config.get(InputFormatConfig.SERIALIZED_TABLE_PREFIX
+ name));
+ Table table =
SerializationUtil.deserializeFromBase64(config.get(InputFormatConfig.SERIALIZED_TABLE_PREFIX
+ name));
+ checkAndSetIoConfig(config, table);
+ return table;
+ }
+
+ /**
+ * If enabled, it populates the FileIO's hadoop configuration with the input
config object.
+ * This might be necessary when the table object was serialized without the
FileIO config.
+ *
+ * @param config Configuration to set for FileIO, if enabled
+ * @param table The Iceberg table object
+ */
+ public static void checkAndSetIoConfig(Configuration config, Table table) {
+ if (config.getBoolean(InputFormatConfig.CONFIG_SERIALIZATION_DISABLED,
false) &&
Review comment:
nit: false should be moved to be
`InputFormatConfig.CONFIG_SERIALIZATION_DISABLED_DEFAULT`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
