rizaon commented on code in PR #4518:
URL: https://github.com/apache/iceberg/pull/4518#discussion_r855599076
##########
core/src/main/java/org/apache/iceberg/CatalogProperties.java:
##########
@@ -53,6 +56,57 @@ private CatalogProperties() {
public static final long CACHE_EXPIRATION_INTERVAL_MS_DEFAULT =
TimeUnit.SECONDS.toMillis(30);
public static final long CACHE_EXPIRATION_INTERVAL_MS_OFF = -1;
+ /**
+ * When using {@link CachingFileIO} as io-impl, define the internal FileIO
that is
+ * wrapped by CachingFileIO.
+ * <p>
+ * Default to {@link HadoopFileIO} if not defined.
+ */
+ public static final String CONTENT_CACHE_WRAPPED_FILE_IO_IMPL =
"cache.content.wrapped-io-impl";
+ public static final String CONTENT_CACHE_WRAPPED_FILE_IO_IMPL_DEFAULT =
HadoopFileIO.class.getName();
Review Comment:
`cache-enabled` is already taken by `CachingCatalog`
https://github.com/apache/iceberg/blob/ed54952b51a6cdc7d2350b8223c276421845a8e2/core/src/main/java/org/apache/iceberg/CatalogProperties.java#L39
Is it OK to enable both `CachingCatalog` and `CachingFileIO` by sharing this
property key? Or should I make new key like `file-cache-enabled=true`?
##########
core/src/main/java/org/apache/iceberg/CatalogProperties.java:
##########
@@ -53,6 +56,57 @@ private CatalogProperties() {
public static final long CACHE_EXPIRATION_INTERVAL_MS_DEFAULT =
TimeUnit.SECONDS.toMillis(30);
public static final long CACHE_EXPIRATION_INTERVAL_MS_OFF = -1;
+ /**
+ * When using {@link CachingFileIO} as io-impl, define the internal FileIO
that is
+ * wrapped by CachingFileIO.
+ * <p>
+ * Default to {@link HadoopFileIO} if not defined.
+ */
+ public static final String CONTENT_CACHE_WRAPPED_FILE_IO_IMPL =
"cache.content.wrapped-io-impl";
+ public static final String CONTENT_CACHE_WRAPPED_FILE_IO_IMPL_DEFAULT =
HadoopFileIO.class.getName();
+
+ /**
+ * Controls the duration for which entries in the {@link CachingFileIO} are
cached.
+ * <p>
+ * Behavior of specific values of cache.expiration-interval-ms:
+ * <ul>
+ * <li> Negative Values - Caching disabled</li>
+ * <li> Zero - Cache entries expires only if it gets evicted due to memory
pressure from
+ * {@link #CONTENT_CACHE_MAX_TOTAL_BYTES} setting.
+ * </li>
+ * <li> Positive Values - Cache entries expire if not accessed via the
cache after this many milliseconds</li>
+ * </ul>
+ */
+ public static final String CONTENT_CACHE_EXPIRATION_INTERVAL_MS =
"cache.content.expiration-interval-ms";
Review Comment:
I see catalog properties already have key `CACHE_ENABLED` and
`CACHE_EXPIRATION_INTERVAL_MS` tied to `CachingCatalog` (which has
`Cache<TableIdentifier, Table>` inside). Thus, I prefix this with `CONTENT_` to
signify that this is different kind of caching. It is caching the file content
(bytes) and not file handle or other java objects.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
