gaborkaszab commented on code in PR #14440:
URL: https://github.com/apache/iceberg/pull/14440#discussion_r2477916221
##########
core/src/main/java/org/apache/iceberg/CachingCatalog.java:
##########
@@ -48,41 +49,62 @@ public class CachingCatalog implements Catalog {
private static final MetadataTableType[] METADATA_TABLE_TYPE_VALUES =
MetadataTableType.values();
public static Catalog wrap(Catalog catalog) {
- return wrap(catalog, CatalogProperties.CACHE_EXPIRATION_INTERVAL_MS_OFF);
+ return wrap(
+ catalog,
+ true,
+ CatalogProperties.CACHE_EXPIRATION_INTERVAL_MS_OFF,
+ CatalogProperties.CACHE_POLICY_DEFAULT);
}
public static Catalog wrap(Catalog catalog, long expirationIntervalMillis) {
- return wrap(catalog, true, expirationIntervalMillis);
+ return wrap(catalog, true, expirationIntervalMillis,
CatalogProperties.CACHE_POLICY_DEFAULT);
+ }
+
+ public static Catalog wrap(Catalog catalog, long expirationIntervalMillis,
String cachePolicy) {
+ return new CachingCatalog(catalog, true, expirationIntervalMillis,
cachePolicy);
}
public static Catalog wrap(
- Catalog catalog, boolean caseSensitive, long expirationIntervalMillis) {
- return new CachingCatalog(catalog, caseSensitive,
expirationIntervalMillis);
+ Catalog catalog, boolean caseSensitive, long expirationIntervalMillis,
String cachePolicy) {
+ return new CachingCatalog(catalog, caseSensitive,
expirationIntervalMillis, cachePolicy);
}
private final Catalog catalog;
private final boolean caseSensitive;
+ private final String cachePolicy;
Review Comment:
nit: cacheExpirationPolicy?
##########
core/src/main/java/org/apache/iceberg/CachingCatalog.java:
##########
@@ -104,15 +126,26 @@ public void onRemoval(TableIdentifier tableIdentifier,
Table table, RemovalCause
}
private Cache<TableIdentifier, Table> createTableCache(Ticker ticker) {
- Caffeine<Object, Object> cacheBuilder = Caffeine.newBuilder().softValues();
+ Caffeine<TableIdentifier, Table> cacheBuilder =
+ Caffeine.newBuilder()
+ .softValues()
+ .removalListener(new MetadataTableInvalidatingRemovalListener())
Review Comment:
Any reason moving these out from the `if` below? Ticker is only relevant
there. I also wouldn't add removalListener, nor executor here
##########
core/src/main/java/org/apache/iceberg/CatalogProperties.java:
##########
@@ -48,6 +49,23 @@ private CatalogProperties() {}
public static final boolean CACHE_CASE_SENSITIVE_DEFAULT = true;
+ /**
+ * Controls policy of caffeine cache
+ *
+ * <p>Behavior of specific values of cache.strategy:
+ *
+ * <ul>
+ * <li>EXPIRE_AFTER_ACCESS - cache entries are never evicted as long as
they are being accessed
+ * frequently
+ * <li>EXPIRE_AFTER_WRITE - cache entries are evicted frequently after
cache write
+ * </ul>
+ */
+ public static final String CACHE_POLICY = "cache.strategy";
Review Comment:
nit: CACHE_EXPIRATION_POLICY, "cache.expiration-policy"
##########
core/src/main/java/org/apache/iceberg/CachingCatalog.java:
##########
@@ -104,15 +126,26 @@ public void onRemoval(TableIdentifier tableIdentifier,
Table table, RemovalCause
}
private Cache<TableIdentifier, Table> createTableCache(Ticker ticker) {
- Caffeine<Object, Object> cacheBuilder = Caffeine.newBuilder().softValues();
+ Caffeine<TableIdentifier, Table> cacheBuilder =
+ Caffeine.newBuilder()
+ .softValues()
+ .removalListener(new MetadataTableInvalidatingRemovalListener())
+ .executor(Runnable::run)
+ .ticker(ticker);
if (expirationIntervalMillis > 0) {
- return cacheBuilder
- .removalListener(new MetadataTableInvalidatingRemovalListener())
- .executor(Runnable::run) // Makes the callbacks to removal listener
synchronous
- .expireAfterAccess(Duration.ofMillis(expirationIntervalMillis))
- .ticker(ticker)
- .build();
+ switch (cachePolicy) {
+ case "EXPIRE_AFTER_WRITE":
+ cacheBuilder =
cacheBuilder.expireAfterWrite(Duration.ofMillis(expirationIntervalMillis));
+ break;
+ case "EXPIRE_AFTER_ACCESS":
+ cacheBuilder =
+
cacheBuilder.expireAfterAccess(Duration.ofMillis(expirationIntervalMillis));
+ break;
+ default:
Review Comment:
Wouldn't be needed if policy was an enum
##########
core/src/test/java/org/apache/iceberg/hadoop/TestCachingCatalog.java:
##########
@@ -181,7 +181,8 @@ public void testNonExistingTable() throws Exception {
@Test
public void testTableWithMetadataTableName() throws Exception {
TestableCachingCatalog catalog =
- TestableCachingCatalog.wrap(hadoopCatalog(), EXPIRATION_TTL, ticker);
+ TestableCachingCatalog.wrap(
Review Comment:
Can't we have a constructor with the old signature that defaults to the
default policy? We could get away not changing these tests then.
##########
core/src/main/java/org/apache/iceberg/CatalogProperties.java:
##########
@@ -48,6 +49,23 @@ private CatalogProperties() {}
public static final boolean CACHE_CASE_SENSITIVE_DEFAULT = true;
+ /**
+ * Controls policy of caffeine cache
+ *
+ * <p>Behavior of specific values of cache.strategy:
+ *
+ * <ul>
+ * <li>EXPIRE_AFTER_ACCESS - cache entries are never evicted as long as
they are being accessed
+ * frequently
+ * <li>EXPIRE_AFTER_WRITE - cache entries are evicted frequently after
cache write
+ * </ul>
+ */
+ public static final String CACHE_POLICY = "cache.strategy";
+
+ public static final List<String> CACHE_POLICY_VALUES =
Review Comment:
This could be an enum
##########
.palantir/revapi.yml:
##########
@@ -404,6 +404,129 @@ acceptedBreaks:
old: "method org.apache.iceberg.orc.ORC.WriteBuilder
org.apache.iceberg.orc.ORC.WriteBuilder::config(java.lang.String,\
\ java.lang.String)"
justification: "Removing deprecations for 1.2.0"
+ "1.10.0":
Review Comment:
I don't think these changes are relevant for this PR. Would you mind taking
a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
