kbendick commented on a change in pull request #3543:
URL: https://github.com/apache/iceberg/pull/3543#discussion_r751665478
##########
File path:
spark/v3.2/spark/src/main/java/org/apache/iceberg/spark/SparkCatalog.java
##########
@@ -384,14 +386,23 @@ public boolean dropNamespace(String[] namespace) throws
NoSuchNamespaceException
@Override
public final void initialize(String name, CaseInsensitiveStringMap options) {
- this.cacheEnabled =
Boolean.parseBoolean(options.getOrDefault("cache-enabled", "true"));
+ this.cacheEnabled = PropertyUtil.propertyAsBoolean(options,
+ CatalogProperties.TABLE_CACHE_ENABLED,
CatalogProperties.TABLE_CACHE_ENABLED_DEFAULT);
+ this.cacheExpirationIntervalMs = PropertyUtil.propertyAsLong(options,
+ CatalogProperties.TABLE_CACHE_EXPIRATION_INTERVAL_MS,
+ CatalogProperties.TABLE_CACHE_EXPIRATION_INTERVAL_MS_DEFAULT);
+
+ Preconditions.checkArgument(cacheEnabled || cacheExpirationIntervalMs > 0L,
+ "The catalog's table cache expiration interval must be set to zero via
the property {} if caching is disabled",
+ CatalogProperties.TABLE_CACHE_EXPIRATION_INTERVAL_MS);
+
Review comment:
I think that instead of throwing, since some of the tests are failing
now where cache is simply disabled, I'm going to check if cache is disabled and
the user explicitly set a value greater than zero for the cache expiration
interval ms.
If the user simply disabled caching and didn't change that field (or set it
to zero), we won't throw in that situation and we will default it to zero
instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
