This is an automated email from the ASF dual-hosted git repository.
dhuo pushed a change to branch persistence-poc
in repository https://gitbox.apache.org/repos/asf/polaris.git
from c7c9c459 Extract BasePersistence interface as parent interface of
PolarisMetaStoreSession; only leave the transaction-specific methods in
PolarisMetaStoreSession
add 3ce3cdb1 Fix small quickstart typo (#1061)
add 9944af7c Enable rat check on md files and fix the existing ones (#1050)
add fce3fafe main: Update dependency boto3 to v1.37.0 (#1052)
add 11cf1754 main: Update dependency software.amazon.awssdk:bom to
v2.30.27 (#1053)
add 839bc257 Integration tests: improve support for temporary folders
(#987)
add e0002b36 build-logic: exclude generated code from errorprone (#1035)
add fb241d30 build-logic: let javadoc depend on jandex (#1034)
add 4db4e80a Add Prashant in the collaborators list (#1064)
add 175d3529 main: Update dependency org.slf4j:slf4j-api to v2.0.17 (#1065)
add f3d9141c Add policies for metadata compaction, orphan file removal and
snapshot retention (#969)
add f0c5f05c Add AWS GOV partitions support (#1056)
add 0cf3cd7c Docs: Add an example to the command-line-interface.md. (#1062)
new db9e71d2 Merge branch 'main' of github.com:dennishuo/polaris into
dhuo-persistence-poc-extract-abstract-base
new 87a8010f Push all evidence of the two-phase lookupEntityByName into
only the transactional-style PolarisMetaStoreSession, so that BasePersistence
properly exposes a lookupEntityByName method where impls that use secondary
indexes can easily just lookup an entity by name instead of doing two lookups.
new 13ba1616 Turn PolarisMetaStoreSession into an abstract class and make
lookupEntityActive protected-visibility; remove all callsites where
PolarisMetaStoreManagerImpl calls it. Technically, while in the same package
this doesn't prevent it from leaking, but we could reposition
PolarisMetaStoreSession into a separate transaction-specific package to help
protect it from leaking the lower-level abstractions.
new 5743dd26 Pushdown all calls to writeToEntities into
PolarisMetaStoreSession, and add writeEntity method to BasePersistence, with a
default impl in PolarisMetaStoreSession containing what was previously in
PolarisMetaStoreManagerImpl. This now protects all writes in
PolarisMetaStoreManagerImpl from dealing with the three-table implementation
detail. Technically slightly changes the ordering of updates within a
transaction for renameEntity, but is arguably a more correct ordering, [...]
new b739fd1a Add originalEntity to the writeEntity method to enable
compare-and-swap behavior from the underlying BasePersistence.
The 5 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.asf.yaml | 1 +
.../src/main/kotlin/polaris-java.gradle.kts | 5 +
build.gradle.kts | 2 -
.../PolarisEclipseLinkMetaStoreSessionImpl.java | 2 +-
gradle/libs.versions.toml | 4 +-
integration-tests/README.md | 19 ++
integration-tests/build.gradle.kts | 1 +
.../service/it/env/IntegrationTestsHelper.java | 57 ++++++
.../it/test/PolarisApplicationIntegrationTest.java | 44 ++---
.../it/test/PolarisRestCatalogIntegrationTest.java | 34 ++--
.../it/test/PolarisSparkIntegrationTest.java | 9 +-
.../service/it/env/IntegrationTestsHelperTest.java | 68 ++++++++
.../polaris/core/persistence/BasePersistence.java | 84 +++++----
.../persistence/PolarisMetaStoreManagerImpl.java | 189 ++++++++------------
.../core/persistence/PolarisMetaStoreSession.java | 148 ++++++++++++++--
.../PolarisTreeMapMetaStoreSessionImpl.java | 2 +-
.../aws/AwsCredentialsStorageIntegration.java | 10 +-
.../storage/aws/AwsStorageConfigurationInfo.java | 25 ++-
.../system/data-compaction/2025-02-03.json | 6 +-
.../2025-02-03.json | 16 +-
.../system/orphan-file-removal/2025-02-03.json | 50 ++++++
.../2025-02-03.json | 18 +-
.../aws/AwsCredentialsStorageIntegrationTest.java | 191 +++++++++++++++------
quarkus/admin/README.md | 21 ++-
quarkus/defaults/README.md | 21 ++-
quarkus/server/README.md | 21 ++-
.../service/quarkus/entity/CatalogEntityTest.java | 5 +-
regtests/client/python/pyproject.toml | 2 +-
.../in-dev/unreleased/command-line-interface.md | 5 +
site/content/in-dev/unreleased/entities.md | 2 +
site/content/in-dev/unreleased/quickstart.md | 2 +-
spec/README.md | 20 +++
32 files changed, 785 insertions(+), 299 deletions(-)
create mode 100644
integration-tests/src/main/java/org/apache/polaris/service/it/env/IntegrationTestsHelper.java
create mode 100644
integration-tests/src/test/java/org/apache/polaris/service/it/env/IntegrationTestsHelperTest.java
copy polaris-core/src/main/resources/schemas/policies/system/{data-compaction
=> metadata-compaction}/2025-02-03.json (64%)
create mode 100644
polaris-core/src/main/resources/schemas/policies/system/orphan-file-removal/2025-02-03.json
copy polaris-core/src/main/resources/schemas/policies/system/{data-compaction
=> snapshot-retention}/2025-02-03.json (64%)
