This is an automated email from the ASF dual-hosted git repository.
yuqi1129 pushed a change to branch feat/12166-occ-drop-cas
in repository https://gitbox.apache.org/repos/asf/gravitino.git
discard 6dee767f6f [#12166] improvement(core): version-checked soft-delete
(drop CAS) for schema (non-cascade)
discard 2b4af4158e [#12166] improvement(core): version-checked soft-delete
(drop CAS) for user, group, role
discard 09c4af9ef5 [#12166] improvement(core): version-checked soft-delete
(drop CAS) for metalake and catalog
discard 0469549218 [#12166] improvement(core): version-checked soft-delete
(drop CAS) for view, function, model
discard 67a412d2a4 [#12166] improvement(core): version-checked soft-delete
(drop CAS) for table and fileset
discard 32b2a823ab [#12166] improvement(core): version-checked soft-delete
(drop CAS) for topic
discard 6779604b61 [#12166] improvement(core): add OCC version columns to
model_meta and wire version CAS
discard e91b088256 [#12166] improvement(core): slim table/function UPDATE
WHERE to version CAS; keep fileset/policy full-row
discard dc35507336 [#12166] improvement(core): slim UPDATE WHERE to version
CAS for metalake/catalog/schema/tag/user/group/role
discard b92ead1eea [#12166] improvement(core): slim topic UPDATE WHERE to
version CAS
discard ed8e5c673a [#12166] improvement(core): always raise current_version on
entity update for OCC
add b464820eb0 [#11844] feat(tag): Support tags for views and functions
(#11897)
add f5948f872b [#11985] fix(openapi): drop stray OpenAPI 3.1 null token
from statistics value (#11987)
add 61ee16ae63 [#11980] fix(core): sort column ordinal positions while
fetching from entity store (#11979)
add fe44f02ecf [#11976] fix(core): refresh built-in job template when
content drifts even if version is unchanged (#11978)
add c2c1b8a672 [#11955] feat(iceberg): native geometry and geography type
support for Iceberg V3 (#11958)
add cf289fb759 [#11590] fix(doris): fix partition parsing for Doris 3.0+
format (#11732)
add 9fbed05f36 [MINOR] fix: polish log and error message quality across
Java modules (#10387)
add 7ba1da7166 [#11962] feat(iceberg): support V3 nanosecond timestamp
types (#11938)
add e71af9ca98 [#11902] feat(client-java): Support tags for views and
functions (#11903)
add 40ab9a58f2 [#11983] test(iceberg): add end-to-end coverage for
round-trip of remaining V2 column types (#11984)
add 425112e554 [MINOR] docs: update Doris catalog documentation for
3.0.x/4.0.x compatibility (#11995)
add 69ef06b81b [#11951] feat(iceberg): map V3 unknown type to Gravitino
NullType (#11969)
add 438da702f1 [#12009] improvement(core): Support custom case sensitivity
normalization in Capability (#12010)
add 771c5e22cd [#11966] feat(auth): add X-Gravitino-Active-Roles header
grammar and parser (#11970)
add 8d6591d9f0 [#11946] improvement(api): Add properties field to
TableChange.AddIndex for custom index parameters (#12013)
add e5fe677dab [#11963] fix(cli): support parsing the variant column type
(#11964)
add 4ff7aac259 [#12014] fix(core): Don't wrap plain ClassNotFoundException
in IsolatedClassLoader (#12015)
add 4059726b7f [#5820] test(client-python): Improve IT server lifecycle
(#11977)
add b473bfd98f [MINOR] chore: Add nevzheng as GitHub triage collaborator
(#12028)
add 41cb729b6e [#3767] refactor(client-python): Share metalake operation
test fixtures (#11993)
add f229b4c427 build(deps): bump cachetools from 7.0.1 to 7.1.4 in
/clients/client-python (#11868)
add ece6c4168b build(deps): update fakeredis requirement from <2.35.0 to
<2.37.0 in /mcp-server (#11866)
add 1f5cee0770 [#3767] refactor(client-python): Share table test fixtures
(#11992)
add 64009ee93c [MINOR] fix(web-v2): Normalize service admins configuration
(#12046)
add 11c39c2d2a [#3767] refactor(client-python): Share metalake integration
test setup (#12005)
add d4190675a0 [#12037] improvement(authz): Improve the JCasbin checks
(#12001)
add 4074abf914 [#11985] fix(openapi): repair three codegen-breaking
defects invisible to lint (#11988)
add c10925adb7 [#12039] feat(client-python): Add view API definitions
(#12040)
add 840fe67f55 Web UI support for built-in IdP Basic login (#11812)
add 40f9cb4323 [MINOR] fix(web-v2): Remove stale twc prefix from toast
(#11950)
add 5863632372 [MINOR] test(client-python): Fix API unit test discovery
(#12041)
add 849aa37c71 [#11985] fix(openapi): restore the ExternalType variant to
DataType (#11991)
add 988bbe0438 [#11985] fix(openapi): clear the remaining non-lineage
recommended-strict findings (#11990)
add b30e457318 [#11985] fix(openapi): correct the OpenLineage translation
in the lineage schemas (#11989)
add da99abcc36 [#11943] improvement(iceberg,core): validate warehouse at
catalog creation and fail fast with an actionable error (#11959)
add 6842f00b74 [MINOR] improvement(cache): Add logs for Caffeine cache
removals (#12043)
add c60b82cc63 [#12038] improvement(scripts): increase relational store
column comment length limit to 4096 (#12047)
add a03467c5e5 [#11967] feat(auth): narrow allow enforcement to active
roles (deny stays global) (#12044)
add fc88ddc8d4 [#12035] improvement(catalog-jdbc): block DBCP2
class-loading properties in datasource config (#12036)
add e675c0465c [#12091] improvement(ci): Route contrib catalog tests to
contrib CI (#12092)
add b49d895eaf [MINOR] improvement(authz): Add debug logging for
list-authorization short-circuit (#12101)
add 23f871f805 [#11802] fix(clickhouse): support user-defined GRANULARITY
and validate shard key type (#11806)
add 648fdd7c1a [#12089] improvement(client-python): Implement client-side
view models (#12090)
add 394e0c6a99 build(deps-dev): bump coverage from 7.13.4 to 7.14.3 in
/clients/client-python (#11872)
add 65d305fdaa [#12097] improvement(core): Validate names before
normalizing in CapabilityHelpers (#12098)
add d9c9237410 [#10093] feat(core): Introduce ClassLoaderPool to share
ClassLoaders across same-type catalogs (#10480)
add afbd958c45 [MINOR] fix(jdbc): Quote comments in generated DDL (#12007)
add 5fe2959078 [#11923] docs(core): add multi-node entity store cache
invalidation design (#11738)
add b0aaac4b9a [#12106] improvement(common): gate Kerberos login-module
debug on sun.security.krb5.debug (#12107)
add 941f030f3e [#12116] improvement(client-python): Add view create/drop
operations (#12117)
add f8d7523b0d [#12095] feat(auth): parse X-Gravitino-Active-Roles into
the request path (#12096)
add 1249341afd [#12102] feat(jdbc-doris): Support frequently-used table
properties (#12103)
add cf1b173ca8 [MINOR] docs: update credential vending, add vended
credentials to Trino (#12165)
add 9e790c649d [#12099] feat(mcp-server): add read tools for table
partitions (#12128)
add c54b342ac8 [#12154] fix(core): Clean up orphaned metadata object
relations (#12160)
add 2214a29dcc [#12158] improvement(client-python): Add view query/alter
operations (#12159)
add a33fa0d1d7 [#12169] fix(auth): clear only role policies on cache
eviction (#12172)
add de7b8a3a0c [#12166] improvement(core): always raise current_version on
entity update for OCC
add 5bf069ada2 [#12166] improvement(core): slim topic UPDATE WHERE to
version CAS
add 313f8cbef1 [#12166] improvement(core): slim UPDATE WHERE to version
CAS for metalake/catalog/schema/tag/user/group/role
add cf8184833d [#12166] improvement(core): slim table/function UPDATE
WHERE to version CAS; keep fileset/policy full-row
add c5ce5c0ea2 [#12166] improvement(core): add OCC version columns to
model_meta and wire version CAS
add 3d7e506910 [#12166] fix(core): roll back dependent writes on OCC
conflicts
add 31ea11df0f [#12166] fix(core): load model OCC versions without cache
new feea8f4c45 [#12166] improvement(core): version-checked soft-delete
(drop CAS) for topic
new 52a5727d49 [#12166] improvement(core): version-checked soft-delete
(drop CAS) for table and fileset
new 76140de5b4 [#12166] improvement(core): version-checked soft-delete
(drop CAS) for view, function, model
new e6245150e7 [#12166] improvement(core): version-checked soft-delete
(drop CAS) for metalake and catalog
new 47525235e2 [#12166] improvement(core): version-checked soft-delete
(drop CAS) for user, group, role
new 897af74faf [#12166] improvement(core): version-checked soft-delete
(drop CAS) for schema (non-cascade)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (6dee767f6f)
\
N -- N -- N refs/heads/feat/12166-occ-drop-cas (897af74faf)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 6 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.asf.yaml | 1 +
.../workflows/backend-integration-test-action.yml | 20 +-
.github/workflows/backend-integration-test.yml | 33 --
.github/workflows/build.yml | 3 +
.github/workflows/contrib-catalog-test.yml | 132 ++++++
.../org/apache/gravitino/function/Function.java | 9 +
.../java/org/apache/gravitino/rel/TableChange.java | 53 ++-
.../main/java/org/apache/gravitino/rel/View.java | 9 +
.../java/org/apache/gravitino/rel/types/Type.java | 10 +
.../java/org/apache/gravitino/rel/types/Types.java | 165 +++++++
.../java/org/apache/gravitino/TestTableChange.java | 101 +++++
.../java/org/apache/gravitino/rel/TestTypes.java | 26 ++
.../ranger/RangerAuthorizationPlugin.java | 16 +-
.../ranger/RangerClientExtension.java | 4 +-
.../authorization/ranger/RangerHelper.java | 2 +-
.../gcs/credential/GCSTokenGenerator.java | 2 +-
.../catalog/clickhouse/ClickHouseConstants.java | 6 +
.../operations/ClickHouseTableOperations.java | 267 +++++++++--
.../test/CatalogClickHouseClusterIT.java | 252 ++++++++++
.../integration/test/CatalogClickHouseIT.java | 215 +++++++++
.../operations/TestClickHouseTableOperations.java | 269 ++++++++++-
.../TestClickHouseTableOperationsCluster.java | 379 ++++++++++++++++
.../catalog/fileset/FilesetCatalogOperations.java | 12 +-
.../catalog/hive/HiveCatalogOperations.java | 2 -
.../catalog/jdbc/JdbcCatalogOperations.java | 17 -
.../catalog/jdbc/utils/DataSourceUtils.java | 56 +++
.../catalog/jdbc/utils/JdbcConnectorUtils.java | 61 +++
.../catalog/jdbc/TestJdbcCatalogOperations.java | 71 +--
.../jdbc/utils/TestDataSourceUrlValidation.java | 257 ++++++++++-
.../catalog/jdbc/utils/TestJdbcConnectorUtils.java | 69 +++
.../doris/DorisTablePropertiesMetadata.java | 44 +-
.../doris/operation/DorisTableOperations.java | 15 +-
.../gravitino/catalog/doris/utils/DorisUtils.java | 19 +-
.../gravitino/catalog/doris/TestDorisCatalog.java | 124 ++++-
.../doris/integration/test/CatalogDoris3xIT.java | 240 ++++++++++
.../doris/integration/test/CatalogDoris4xIT.java | 241 ++++++++++
.../doris/integration/test/CatalogDorisIT.java | 129 ++++++
.../operation/TestDorisDatabaseOperations.java | 2 +-
.../TestDorisDatabaseOperationsSqlGeneration.java} | 24 +-
.../TestDorisTableOperationsSqlGeneration.java | 63 ++-
.../catalog/doris/utils/TestDorisUtils.java | 42 ++
.../mysql/operation/MysqlTableOperations.java | 12 +-
.../TestMysqlTableOperationsSqlGeneration.java | 60 ++-
.../operation/PostgreSqlSchemaOperations.java | 5 +-
.../operation/PostgreSqlTableOperations.java | 20 +-
...stPostgreSqlSchemaOperationsSqlGeneration.java} | 27 +-
...TestPostgreSqlTableOperationsSqlGeneration.java | 59 ++-
.../operations/StarRocksTableOperations.java | 22 +-
.../catalog/starrocks/utils/StarRocksUtils.java | 7 +-
.../operation/TestStarRocksTableOperations.java | 50 ++
.../TestStarRocksTableOperationsSqlGeneration.java | 77 +++-
.../starrocks/utils/TestStarRocksUtils.java | 25 +
.../catalog/lakehouse/iceberg/IcebergCatalog.java | 16 +
.../iceberg/IcebergCatalogOperations.java | 49 +-
.../iceberg/converter/FromIcebergType.java | 22 +
.../lakehouse/iceberg/converter/ToIcebergType.java | 57 ++-
.../iceberg/converter/ToIcebergTypeVisitor.java | 7 +
.../lakehouse/iceberg/TestIcebergCatalog.java | 11 +
.../iceberg/TestIcebergCatalogOperations.java | 53 +++
.../iceberg/converter/TestConvertUtil.java | 140 ++++++
.../integration/test/CatalogIcebergBaseIT.java | 364 +++++++++++++--
.../integration/test/CatalogIcebergRestIT.java | 130 ++++++
.../integration/test/IcebergClassLoaderPoolIT.java | 265 +++++++++++
.../integration/test/TestMultipleJDBCLoad.java | 12 +-
.../lakehouse/paimon/PaimonCatalogOperations.java | 4 +-
.../lakehouse/paimon/ops/PaimonCatalogOps.java | 2 +-
.../lakehouse/paimon/utils/CatalogUtils.java | 6 +-
.../catalog/hadoop/auth/KerberosAuthUtils.java | 4 +-
.../apache/gravitino/hive/CachedClientPool.java | 10 +-
.../org/apache/gravitino/cli/TypeConverter.java | 2 +
.../org/apache/gravitino/cli/TestParseType.java | 6 +
.../apache/gravitino/cli/TestTypeConverter.java | 1 +
.../org/apache/gravitino/client/DTOConverters.java | 3 +-
.../org/apache/gravitino/client/ErrorHandlers.java | 3 +
.../client/FunctionCatalogOperations.java | 10 +-
.../{GenericColumn.java => GenericFunction.java} | 97 ++--
.../{GenericColumn.java => GenericView.java} | 105 +++--
.../gravitino/client/KerberosTokenProvider.java | 4 +-
.../apache/gravitino/client/RelationalCatalog.java | 6 +-
.../gravitino/client/TestFunctionCatalog.java | 46 ++
.../apache/gravitino/client/TestSupportTags.java | 107 +++++
.../gravitino/client/integration/test/TagIT.java | 227 +++++++++
clients/client-python/build.gradle.kts | 19 +-
clients/client-python/gravitino/api/catalog.py | 10 +
.../client-python/gravitino/api/rel/dialects.py | 9 +-
.../rel/representation.py} | 12 +-
.../rel/sql_representation.py} | 35 +-
.../rel/types/json_serdes/_helper/serdes_utils.py | 24 +-
.../client-python/gravitino/api/rel/types/type.py | 8 +
.../client-python/gravitino/api/rel/types/types.py | 129 ++++++
clients/client-python/gravitino/api/rel/view.py | 68 +++
.../gravitino/api/rel/view_catalog.py | 149 ++++++
.../client-python/gravitino/api/rel/view_change.py | 214 +++++++++
.../client-python/gravitino/client/generic_view.py | 63 +++
.../gravitino/client/relational_catalog.py | 224 ++++++++-
.../dto/rel/json_serdes/representation_serdes.py | 49 ++
.../auditable.py => dto/rel/representation_dto.py} | 17 +-
.../sql_representation_dto.py} | 40 +-
.../client-python/gravitino/dto/rel/view_dto.py | 105 +++++
.../gravitino/dto/requests/view_create_request.py | 96 ++++
.../gravitino/dto/requests/view_update_request.py | 166 +++++++
..._updates_request.py => view_updates_request.py} | 26 +-
...{function_list_response.py => view_response.py} | 34 +-
.../gravitino/dto/util/dto_converters.py | 54 ++-
clients/client-python/gravitino/exceptions/base.py | 8 +
...able_error_handler.py => view_error_handler.py} | 20 +-
clients/client-python/gravitino/utils/serdes.py | 10 +
clients/client-python/requirements-dev.txt | 4 +-
clients/client-python/requirements.txt | 2 +-
.../tests/integration/integration_test_env.py | 78 +++-
.../tests/integration/test_catalog.py | 37 +-
.../tests/integration/test_group_management.py | 10 +-
.../client-python/tests/integration/test_owner.py | 37 +-
.../tests/integration/test_relational_catalog.py | 171 +++++++
.../tests/integration/test_role_management.py | 10 +-
.../client-python/tests/integration/test_schema.py | 32 +-
.../client-python/tests/integration/test_user.py | 41 +-
.../{gravitino => tests/unittests}/api/__init__.py | 0
.../unittests/api/rel/test_sql_representation.py | 54 +++
.../tests/unittests/api/rel/test_types.py | 26 ++
.../tests/unittests/api/rel/test_view.py | 69 +++
.../tests/unittests/api/rel/test_view_change.py | 150 ++++++
.../api/stats/test_supports_statistics.py | 9 +-
.../api => tests/unittests/api/tag}/__init__.py | 0
.../unittests/client/operation_test_fixtures.py | 89 ++++
.../client/test_metalake_group_operations.py | 73 +--
.../client/test_metalake_role_operations.py | 167 ++-----
.../client/test_metalake_user_operations.py | 73 +--
.../dto/rel/test_representation_serdes.py | 54 +++
.../dto/rel/test_sql_representation_dto.py | 55 +++
.../tests/unittests/dto/rel/test_table_dto.py | 129 +-----
.../tests/unittests/dto/rel/test_view_dto.py | 293 ++++++++++++
.../dto/requests/test_view_create_request.py | 144 ++++++
.../dto/requests/test_view_update_request.py | 197 ++++++++
.../unittests/dto/responses/test_responses.py | 163 +------
.../unittests/dto/responses/test_view_response.py | 51 +++
.../unittests/dto/util/test_dto_converters.py | 181 +++-----
.../api => tests/unittests/fixtures}/__init__.py | 0
.../tests/unittests/fixtures/table_fixtures.py | 222 +++++++++
.../unittests/json_serdes/test_type_serdes.py | 9 +
.../tests/unittests/test_error_handler.py | 83 ++++
.../tests/unittests/test_generic_view.py | 61 +++
.../tests/unittests/test_relational_catalog.py | 380 +++++++++++-----
.../tests/unittests/test_relational_table.py | 158 +------
.../client-python/tests/unittests/test_requests.py | 133 +-----
.../filesystem/hadoop/BaseGVFSOperations.java | 20 +-
.../org/apache/gravitino/auth/ActiveRoles.java | 164 +++++++
.../apache/gravitino/auth/ActiveRolesParser.java | 122 +++++
.../org/apache/gravitino/auth/AuthConstants.java | 10 +
...rType.java => IllegalActiveRolesException.java} | 33 +-
.../org/apache/gravitino/auth/KerberosUtils.java | 6 +-
.../gravitino/dto/requests/TableUpdateRequest.java | 20 +-
.../java/org/apache/gravitino/json/JsonUtils.java | 19 +
.../org/apache/gravitino/auth/TestActiveRoles.java | 102 +++++
.../gravitino/auth/TestActiveRolesParser.java | 122 +++++
.../org/apache/gravitino/json/TestJsonUtils.java | 33 ++
conf/gravitino.conf.template | 4 +
.../main/java/org/apache/gravitino/Configs.java | 12 +
.../java/org/apache/gravitino/UserPrincipal.java | 32 ++
.../apache/gravitino/audit/AuditLogManager.java | 9 +-
.../authorization/AuthorizationRequestContext.java | 37 ++
.../authorization/GravitinoAuthorizer.java | 2 +-
.../gravitino/authorization/OwnerManager.java | 7 +-
.../auxiliary/AuxiliaryServiceManager.java | 4 +-
.../gravitino/cache/CaffeineEntityCache.java | 4 +-
.../gravitino/catalog/CapabilityHelpers.java | 54 ++-
.../apache/gravitino/catalog/CatalogManager.java | 202 +++++++--
.../catalog/FilesetNormalizeDispatcher.java | 17 +-
.../catalog/FunctionNormalizeDispatcher.java | 17 +-
.../gravitino/catalog/ManagedTableOperations.java | 8 +-
.../catalog/ModelNormalizeDispatcher.java | 17 +-
.../catalog/PartitionNormalizeDispatcher.java | 22 +-
.../catalog/SchemaNormalizeDispatcher.java | 19 +-
.../catalog/SchemaOperationDispatcher.java | 2 +-
.../catalog/TableNormalizeDispatcher.java | 17 +-
.../catalog/TableOperationDispatcher.java | 2 +-
.../catalog/TopicNormalizeDispatcher.java | 17 +-
.../catalog/TopicOperationDispatcher.java | 2 +-
.../gravitino/catalog/ViewNormalizeDispatcher.java | 16 +-
.../gravitino/catalog/ViewOperationDispatcher.java | 2 +-
.../apache/gravitino/connector/BaseCatalog.java | 9 +
.../gravitino/connector/capability/Capability.java | 20 +
.../gravitino/credential/CredentialCache.java | 5 +-
.../gravitino/hook/CatalogHookDispatcher.java | 4 +-
.../job/BuiltInJobTemplateEventListener.java | 7 +-
.../gravitino/listener/AsyncQueueListener.java | 2 +-
.../apache/gravitino/metrics/MetricsSystem.java | 2 +-
.../storage/LancePartitionStatisticStorage.java | 4 +
.../gravitino/storage/relational/JDBCBackend.java | 10 +-
.../relational/RelationalGarbageCollector.java | 29 +-
...r.java => SupportsOrphanedRelationCleanup.java} | 20 +-
.../OrphanedMetadataObjectRelationMapper.java | 81 ++++
.../OrphanedMetadataObjectRelationSQLProvider.java | 96 ++++
.../provider/DefaultMapperPackageProvider.java | 2 +
.../provider/base/ModelMetaBaseSQLProvider.java | 8 +-
.../base/TagMetadataObjectRelBaseSQLProvider.java | 28 ++
.../TagMetadataObjectRelPostgreSQLProvider.java | 28 ++
.../relational/service/FunctionMetaService.java | 27 +-
.../relational/service/GroupMetaService.java | 23 +-
.../OrphanedMetadataObjectRelationService.java | 153 +++++++
.../relational/service/PolicyMetaService.java | 26 +-
.../relational/service/RoleMetaService.java | 21 +-
.../relational/service/TableMetaService.java | 24 +-
.../relational/service/UserMetaService.java | 43 +-
.../java/org/apache/gravitino/tag/TagManager.java | 4 +-
.../org/apache/gravitino/utils/ClassLoaderKey.java | 70 +++
.../apache/gravitino/utils/ClassLoaderPool.java | 216 +++++++++
.../gravitino/utils/IsolatedClassLoader.java | 43 +-
.../gravitino/utils/PooledClassLoaderEntry.java | 101 +++++
.../org/apache/gravitino/utils/PrincipalUtils.java | 6 +-
.../java/org/apache/gravitino/TestCatalog.java | 7 +
.../TestAuthorizationRequestContext.java | 17 +
.../gravitino/catalog/TestCapabilityHelpers.java | 145 ++++++
.../gravitino/catalog/TestCatalogManager.java | 74 +++
.../catalog/TestClassLoaderPoolIntegration.java | 349 ++++++++++++++
.../catalog/TestManagedTableOperations.java | 42 ++
.../catalog/TestPartitionNormalizeDispatcher.java | 72 +++
.../catalog/TestTableNormalizeDispatcher.java | 82 ++++
.../gravitino/connector/TestCatalogOperations.java | 10 +-
.../connector/capability/TestCapability.java | 8 +
.../job/TestBuiltInJobTemplateEventListener.java | 52 ++-
.../relational/TestRelationalGarbageCollector.java | 72 +++
.../relational/service/TestCatalogMetaService.java | 157 +++++++
.../service/TestFunctionMetaService.java | 108 +++++
.../relational/service/TestGroupMetaService.java | 108 +++++
.../relational/service/TestModelMetaService.java | 48 ++
.../TestOrphanedMetadataObjectRelationService.java | 139 ++++++
.../relational/service/TestPolicyMetaService.java | 90 ++++
.../relational/service/TestRoleMetaService.java | 96 ++++
.../relational/service/TestSchemaMetaService.java | 146 ++++++
.../relational/service/TestTableMetaService.java | 102 +++++
.../relational/service/TestUserMetaService.java | 155 +++++++
.../relational/service/TestViewMetaService.java | 41 ++
.../org/apache/gravitino/tag/TestTagManager.java | 167 ++++++-
.../gravitino/utils/TestClassLoaderPool.java | 428 +++++++++++++++++
.../gravitino/utils/TestIsolatedClassLoader.java | 12 +
.../gravitino/utils/TestMetadataObjectUtil.java | 11 +
.../gravitino-entity-cache-multinode-design.md | 429 +++++++++++++++++
design-docs/images/coherence-gate.png | Bin 0 -> 86977 bytes
design-docs/images/entity-key-invalidation.png | Bin 0 -> 125578 bytes
design-docs/images/roadmap.png | Bin 0 -> 76171 bytes
design-docs/images/shared-cache.png | Bin 0 -> 63940 bytes
docs/gravitino-mcp-server.md | 2 +
docs/gravitino-server-config.md | 1 +
docs/iceberg-rest-engine/trino.md | 320 +++++++------
docs/jdbc-clickhouse-catalog.md | 12 +-
docs/jdbc-doris-catalog.md | 205 +++++++--
docs/lakehouse-iceberg-catalog.md | 16 +-
docs/manage-relational-metadata-using-gravitino.md | 15 +
docs/manage-tags-in-gravitino.md | 10 +-
docs/open-api/catalogs.yaml | 49 ++
docs/open-api/datatype.yaml | 74 ++-
docs/open-api/filesets.yaml | 1 -
docs/open-api/indexes.yaml | 8 +
docs/open-api/lineage.yaml | 11 +-
docs/open-api/models.yaml | 3 +-
docs/open-api/openapi.yaml | 4 +-
docs/open-api/policies.yaml | 1 +
docs/open-api/statistics.yaml | 6 +-
docs/open-api/tables.yaml | 15 +-
docs/open-api/tags.yaml | 4 +-
docs/security/credential-vending.md | 505 +++++++++++++--------
...partitioning-distribution-sort-order-indexes.md | 13 +-
.../iceberg/common/ops/IcebergCatalogWrapper.java | 74 ---
.../common/utils/IcebergHiveCachedClientPool.java | 9 +-
.../iceberg/service/CatalogWrapperForREST.java | 5 -
.../service/IcebergCatalogWrapperManager.java | 5 +-
.../iceberg/service/IcebergExceptionMapper.java | 1 +
.../service/TestIcebergAuthenticationFilter.java | 20 +
.../integration/test/util/TestDatabaseName.java | 14 +
.../lance/service/LanceAuthenticationFilter.java | 7 +
.../mcp_server/client/gravitino_operation.py | 11 +
.../{topic_operation.py => partition_operation.py} | 63 +--
.../client/plain/plain_rest_client_operation.py | 9 +
.../plain/plain_rest_client_partition_operation.py | 65 +++
mcp-server/mcp_server/tools/__init__.py | 2 +
mcp-server/mcp_server/tools/partition.py | 119 +++++
mcp-server/pyproject.toml | 2 +-
mcp-server/tests/unit/client/test_url_encoding.py | 31 ++
mcp-server/tests/unit/tools/mock_operation.py | 30 ++
mcp-server/tests/unit/tools/test_partition.py | 92 ++++
.../apache/gravitino/idp/IdpUserGroupManager.java | 2 +-
scripts/h2/schema-2.0.0-h2.sql | 2 +-
scripts/h2/upgrade-1.3.0-to-2.0.0-h2.sql | 3 +
scripts/mysql/schema-2.0.0-mysql.sql | 2 +-
scripts/mysql/upgrade-1.3.0-to-2.0.0-mysql.sql | 3 +
scripts/postgresql/schema-2.0.0-postgresql.sql | 2 +-
.../upgrade-1.3.0-to-2.0.0-postgresql.sql | 3 +
.../authentication/AuthenticationFilter.java | 24 +-
.../authentication/KerberosAuthenticator.java | 4 +-
.../authorization/GravitinoAuthorizerProvider.java | 2 +-
.../server/authorization/MetadataAuthzHelper.java | 72 ++-
.../authorization/jcasbin/JcasbinAuthorizer.java | 248 +++++++---
.../jcasbin/JcasbinLoadedRolesCache.java | 22 +-
.../apache/gravitino/server/web/JettyServer.java | 14 +-
.../authentication/TestAuthenticationFilter.java | 80 ++++
.../authentication/TestKerberosAuthenticator.java | 4 +-
.../jcasbin/TestJcasbinAuthorizer.java | 446 +++++++++++++++++-
.../server/web/rest/ExceptionHandlers.java | 10 +
.../gravitino/server/web/rest/OwnerOperations.java | 4 +-
.../server/web/rest/PermissionOperations.java | 4 +-
.../server/web/rest/SchemaOperations.java | 2 +-
.../web/rest/TestMetadataObjectTagOperations.java | 262 ++++++++++-
.../gravitino/trino/connector/GravitinoConfig.java | 32 +-
.../trino/connector/GravitinoConnectorFactory.java | 9 +-
.../connector/GravitinoConnectorPluginManager.java | 14 +-
.../gravitino/trino/connector/GravitinoHandle.java | 4 +-
.../trino/connector/catalog/CatalogRegister.java | 9 +-
web-v2/web/src/app/login/components/BasicLogin.js | 103 +++++
.../web/src/app/login/components/DefaultLogin.js | 78 ++--
web-v2/web/src/app/login/components/SimpleLogin.js | 55 +++
web-v2/web/src/app/login/page.js | 48 +-
web-v2/web/src/app/metalakes/page.js | 3 +-
web-v2/web/src/app/rootLayout/UserSetting.js | 3 +-
web-v2/web/src/components/StyledToast.js | 6 +-
web-v2/web/src/lib/api/auth/index.js | 14 +
web-v2/web/src/lib/provider/session.js | 8 +
web-v2/web/src/lib/store/auth/index.js | 71 ++-
web-v2/web/src/lib/utils/axios/index.js | 30 +-
.../web/src/lib/utils/serviceAdmins.js | 13 +-
320 files changed, 16784 insertions(+), 2849 deletions(-)
create mode 100644 .github/workflows/contrib-catalog-test.yml
create mode 100644
catalogs/catalog-jdbc-common/src/test/java/org/apache/gravitino/catalog/jdbc/utils/TestJdbcConnectorUtils.java
copy
catalogs/{catalog-lakehouse-hudi/src/test/java/org/apache/gravitino/catalog/lakehouse/hudi/backend/hms/TestHudiHMSBackend.java
=>
catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/operation/TestDorisDatabaseOperationsSqlGeneration.java}
(60%)
copy
catalogs/{catalog-lakehouse-hudi/src/test/java/org/apache/gravitino/catalog/lakehouse/hudi/backend/hms/TestHudiHMSBackend.java
=>
catalog-jdbc-postgresql/src/test/java/org/apache/gravitino/catalog/postgresql/operation/TestPostgreSqlSchemaOperationsSqlGeneration.java}
(56%)
create mode 100644
catalogs/catalog-lakehouse-iceberg/src/test/java/org/apache/gravitino/catalog/lakehouse/iceberg/integration/test/IcebergClassLoaderPoolIT.java
copy
clients/client-java/src/main/java/org/apache/gravitino/client/{GenericColumn.java
=> GenericFunction.java} (57%)
copy
clients/client-java/src/main/java/org/apache/gravitino/client/{GenericColumn.java
=> GenericView.java} (58%)
copy mcp-server/mcp_server/client/plain/exception.py =>
clients/client-python/gravitino/api/rel/dialects.py (84%)
copy clients/client-python/gravitino/{auth/auth_constants.py =>
api/rel/representation.py} (77%)
copy clients/client-python/gravitino/{dto/requests/user_add_request.py =>
api/rel/sql_representation.py} (55%)
create mode 100644 clients/client-python/gravitino/api/rel/view.py
create mode 100644 clients/client-python/gravitino/api/rel/view_catalog.py
create mode 100644 clients/client-python/gravitino/api/rel/view_change.py
create mode 100644 clients/client-python/gravitino/client/generic_view.py
create mode 100644
clients/client-python/gravitino/dto/rel/json_serdes/representation_serdes.py
copy clients/client-python/gravitino/{api/auditable.py =>
dto/rel/representation_dto.py} (70%)
copy clients/client-python/gravitino/dto/{responses/table_response.py =>
rel/sql_representation_dto.py} (50%)
create mode 100644 clients/client-python/gravitino/dto/rel/view_dto.py
create mode 100644
clients/client-python/gravitino/dto/requests/view_create_request.py
create mode 100644
clients/client-python/gravitino/dto/requests/view_update_request.py
copy clients/client-python/gravitino/dto/requests/{metalake_updates_request.py
=> view_updates_request.py} (61%)
copy clients/client-python/gravitino/dto/responses/{function_list_response.py
=> view_response.py} (55%)
copy
clients/client-python/gravitino/exceptions/handlers/{table_error_handler.py =>
view_error_handler.py} (83%)
copy clients/client-python/{gravitino => tests/unittests}/api/__init__.py
(100%)
create mode 100644
clients/client-python/tests/unittests/api/rel/test_sql_representation.py
create mode 100644 clients/client-python/tests/unittests/api/rel/test_view.py
create mode 100644
clients/client-python/tests/unittests/api/rel/test_view_change.py
copy clients/client-python/{gravitino/api =>
tests/unittests/api/tag}/__init__.py (100%)
create mode 100644
clients/client-python/tests/unittests/client/operation_test_fixtures.py
create mode 100644
clients/client-python/tests/unittests/dto/rel/test_representation_serdes.py
create mode 100644
clients/client-python/tests/unittests/dto/rel/test_sql_representation_dto.py
create mode 100644
clients/client-python/tests/unittests/dto/rel/test_view_dto.py
create mode 100644
clients/client-python/tests/unittests/dto/requests/test_view_create_request.py
create mode 100644
clients/client-python/tests/unittests/dto/requests/test_view_update_request.py
create mode 100644
clients/client-python/tests/unittests/dto/responses/test_view_response.py
copy clients/client-python/{gravitino/api =>
tests/unittests/fixtures}/__init__.py (100%)
create mode 100644
clients/client-python/tests/unittests/fixtures/table_fixtures.py
create mode 100644 clients/client-python/tests/unittests/test_generic_view.py
create mode 100644
common/src/main/java/org/apache/gravitino/auth/ActiveRoles.java
create mode 100644
common/src/main/java/org/apache/gravitino/auth/ActiveRolesParser.java
copy common/src/main/java/org/apache/gravitino/auth/{AuthenticatorType.java =>
IllegalActiveRolesException.java} (52%)
create mode 100644
common/src/test/java/org/apache/gravitino/auth/TestActiveRoles.java
create mode 100644
common/src/test/java/org/apache/gravitino/auth/TestActiveRolesParser.java
copy
core/src/main/java/org/apache/gravitino/storage/relational/{EntityChangeLogListener.java
=> SupportsOrphanedRelationCleanup.java} (55%)
create mode 100644
core/src/main/java/org/apache/gravitino/storage/relational/mapper/OrphanedMetadataObjectRelationMapper.java
create mode 100644
core/src/main/java/org/apache/gravitino/storage/relational/mapper/OrphanedMetadataObjectRelationSQLProvider.java
create mode 100644
core/src/main/java/org/apache/gravitino/storage/relational/service/OrphanedMetadataObjectRelationService.java
create mode 100644
core/src/main/java/org/apache/gravitino/utils/ClassLoaderKey.java
create mode 100644
core/src/main/java/org/apache/gravitino/utils/ClassLoaderPool.java
create mode 100644
core/src/main/java/org/apache/gravitino/utils/PooledClassLoaderEntry.java
create mode 100644
core/src/test/java/org/apache/gravitino/catalog/TestCapabilityHelpers.java
create mode 100644
core/src/test/java/org/apache/gravitino/catalog/TestClassLoaderPoolIntegration.java
create mode 100644
core/src/test/java/org/apache/gravitino/storage/relational/TestRelationalGarbageCollector.java
create mode 100644
core/src/test/java/org/apache/gravitino/storage/relational/service/TestOrphanedMetadataObjectRelationService.java
create mode 100644
core/src/test/java/org/apache/gravitino/utils/TestClassLoaderPool.java
create mode 100644 design-docs/gravitino-entity-cache-multinode-design.md
create mode 100644 design-docs/images/coherence-gate.png
create mode 100644 design-docs/images/entity-key-invalidation.png
create mode 100644 design-docs/images/roadmap.png
create mode 100644 design-docs/images/shared-cache.png
mode change 100644 => 100755 docs/security/credential-vending.md
copy mcp-server/mcp_server/client/{topic_operation.py =>
partition_operation.py} (57%)
create mode 100644
mcp-server/mcp_server/client/plain/plain_rest_client_partition_operation.py
create mode 100644 mcp-server/mcp_server/tools/partition.py
create mode 100644 mcp-server/tests/unit/tools/test_partition.py
create mode 100644 web-v2/web/src/app/login/components/BasicLogin.js
create mode 100644 web-v2/web/src/app/login/components/SimpleLogin.js
copy web/web/postcss.config.js => web-v2/web/src/lib/utils/serviceAdmins.js
(78%)
