wsxuehao opened a new issue, #12164:
URL: https://github.com/apache/gravitino/issues/12164
### Describe the feature
Gravitino currently relies on package-based classification, such as
isCatalogClass() and isSharedClass(), to determine whether a class should be
loaded by the isolated catalog classloader or delegated to the server
classloader.
This approach works for known package prefixes, but it is fragile because
every newly introduced or relocated package must be explicitly added to the
classification rules. If a package is omitted, catalog or plugin classes may
unexpectedly be loaded by the server classloader.
In addition, ordinary catalog-private third-party dependencies may still
follow parent-first delegation and can therefore be shadowed by dependencies
already present in the Gravitino server classpath.
This feature proposes introducing a child-first loading strategy for
catalog-private classes and dependencies while retaining parent-first loading
for explicitly shared APIs.
### Motivation
Several classloading issues have already been caused by incomplete package
classification, where catalog or plugin classes were unexpectedly delegated to
the server classloader.
The current implementation requires package-prefix rules to be updated
whenever catalog packages are added, moved, or refactored. This creates
recurring maintenance work and makes isolation dependent on developers
remembering to update isCatalogClass().
The current behavior can also cause dependency shadowing. For example, if
both the server and a catalog contain different versions of the same
third-party library, the server version may be loaded first even when the
dependency is intended to be private to the catalog.
This may result in runtime failures such as:
NoClassDefFoundError
NoSuchMethodError
ClassCastException
LinkageError
A child-first strategy for catalog-private dependencies would provide
stronger and more predictable isolation while preserving shared Gravitino API
compatibility.
### Describe the solution
The current catalog-private branch eventually delegates to
URLClassLoader.loadClass():
if (!isSharedClass(name)) {
return super.loadClass(name, resolve);
}
However, super.loadClass() follows the standard parent-first delegation
model. Therefore, even classes intended to be isolated may first be resolved
from the parent classloader.
A possible implementation is to try the catalog classloader first for
non-shared classes and fall back to the parent only when the class is not
available locally.
For example:
```
@Override
protected Class<?> loadClass(String name, boolean resolve)
throws ClassNotFoundException {
synchronized (getClassLoadingLock(name)) {
Class<?> loadedClass = findLoadedClass(name);
if (loadedClass != null) {
return loadedClass;
}
if (isSharedClass(name)) {
return super.loadClass(name, resolve);
}
try {
Class<?> clazz = findClass(name);
if (resolve) {
resolveClass(clazz);
}
return clazz;
} catch (ClassNotFoundException e) {
return super.loadClass(name, resolve);
}
}
}
```
The intended delegation order would become:
Shared classes:
parent/base classloader
↓
catalog classloader if necessary
Catalog-private classes:
catalog classloader
↓
parent classloader as fallback
Parent-first loading should remain enabled for explicitly shared packages,
including:
java.*
javax.*
jdk.*
sun.*
Gravitino public API and SPI packages
logging APIs such as:
org.slf4j.*
org.apache.logging.*
Catalog implementation classes and their private third-party dependencies
would use child-first loading by default.
The existing barrier-class handling could remain unchanged if it is still
required for classes that must always be defined by the isolated classloader.
For backward compatibility, the loading policy could optionally be
controlled by a configuration property, for example:
gravitino.catalog.classloader.strategy=child-first
Possible values could be:
parent-first
child-first
However, if compatibility risk is considered low, child-first could become
the default only for non-shared catalog dependencies without introducing a new
configuration option.
The required code change should be relatively small because the existing
CustomURLClassLoader already contains class classification and synchronization
logic. The main change is replacing the parent-first super.loadClass() path for
catalog-private classes with an explicit findClass()-first path.
### Additional context
Recent classloading issues caused by newly introduced or relocated packages
show that package-prefix-based isolation is difficult to maintain over time.
Adding more package prefixes fixes individual occurrences but does not
address the general problem. Similar failures may recur whenever a new catalog,
authorization plugin, or internal package is introduced.
A child-first strategy for catalog-private dependencies would:
Reduce reliance on isCatalogClass() package-prefix maintenance.
Prevent server dependencies from unexpectedly shadowing catalog-private
dependencies.
Improve isolation between different catalog implementations.
Reduce recurring NoClassDefFoundError and linkage-related failures.
Preserve type compatibility by continuing to load Gravitino public APIs from
the shared parent classloader.
This proposal does not require making every class child-first. It only
changes the default behavior for classes that are not explicitly classified as
shared.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
