CodingCat commented on a change in pull request #3663:
URL: https://github.com/apache/iceberg/pull/3663#discussion_r774213587
##########
File path: core/src/main/java/org/apache/iceberg/hadoop/HadoopCatalog.java
##########
@@ -105,6 +108,8 @@ public void initialize(String name, Map<String, String>
properties) {
this.fileIO = fileIOImpl == null ? new HadoopFileIO(conf) :
CatalogUtil.loadFileIO(fileIOImpl, properties, conf);
this.suppressPermissionError =
Boolean.parseBoolean(properties.get(HADOOP_SUPPRESS_PERMISSION_ERROR));
+
+ this.lockManager = LockManagers.from(properties);
Review comment:
@jackye1995 I think users can still use HadoopCatalog with S3 or local
file system even that's not the optimal solution, so the data loss issue is
still there?
##########
File path: core/src/main/java/org/apache/iceberg/util/LockManager.java
##########
@@ -17,14 +17,14 @@
* under the License.
*/
-package org.apache.iceberg.aws.glue;
+package org.apache.iceberg.util;
Review comment:
done
##########
File path: core/src/main/java/org/apache/iceberg/hadoop/HadoopTables.java
##########
@@ -194,8 +203,24 @@ TableOperations newTableOps(String location) {
if (location.contains(METADATA_JSON)) {
return new StaticTableOperations(location, new HadoopFileIO(conf));
} else {
- return new HadoopTableOperations(new Path(location), new
HadoopFileIO(conf), conf);
+ return new HadoopTableOperations(new Path(location), new
HadoopFileIO(conf), conf,
+ createOrGetLockManager(location));
+ }
+ }
+
+ private LockManager createOrGetLockManager(String location) {
+ Map<String, String> properties = Maps.newHashMap();
+ Iterator<Map.Entry<String, String>> configEntries = conf.iterator();
+ while (configEntries.hasNext()) {
+ Map.Entry<String, String> entry = configEntries.next();
+ String key = entry.getKey();
+ if (key.startsWith("iceberg.lock.")) {
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
