LuciferYang opened a new pull request, #13851:
URL: https://github.com/apache/gravitino/pull/13851

   ### What changes were proposed in this pull request?
   
   With the fileset metadata cache enabled, the Java and Python GVFS clients 
now take the catalog, schema and fileset from the cache when they merge the 
filesystem properties, instead of loading all three from the server on every 
operation.
   
   - Java: `BaseGVFSOperations.getAllProperties` goes through 
`FilesetMetadataCache`. The catalog cache now stores the `Catalog`, and 
`getFilesetCatalog` derives the fileset view from it. The schema loader reuses 
the cached catalog. The loader still rejects non-fileset catalogs, so they are 
not cached.
   - Python: `_merge_fileset_properties` goes through `_get_fileset_catalog`, 
`_get_fileset_schema` and `_get_fileset`.
   
   Secrets and static catalog credentials are not cached and are still fetched 
on every call. With the cache disabled, the loads are unchanged.
   
   ### Why are the changes needed?
   
   #12572 made both clients load the catalog, schema and fileset directly from 
the client while merging properties. Before that change they came from the 
cache-aware accessors, so since #12572, enabling 
`fs.gravitino.filesetMetadataCache.cache.enable` (Java) or 
`enable_fileset_metadata_cache` (Python) no longer removes these three requests 
from each operation. #12572 is not in a release yet.
   
   Fix: #13850
   
   ### Does this PR introduce _any_ user-facing change?
   
   No. With the cache enabled, catalog, schema and fileset properties used for 
the filesystem configuration are cached like the rest of the fileset metadata, 
as they were before #12572.
   
   ### How was this patch tested?
   
   - 
`TestBaseGVFSOperationsSecrets.testMetadataCacheAvoidsRepeatedMetadataLoads` 
(Java) and `test_metadata_cache_avoids_repeated_metadata_loads` (Python): with 
the cache enabled, a second property merge loads nothing again while secrets 
are fetched twice.
   - `TestFilesetMetadataCache.testNonFilesetCatalogIsNotCached`: a non-fileset 
catalog is loaded again on the next call instead of being cached.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to