JingsongLi opened a new pull request, #8066:
URL: https://github.com/apache/paimon/pull/8066
## Summary
Optimize deletion vector index scanning during snapshot planning by avoiding
unnecessary manifest reads and limiting DV index scans to the requested
partition-bucket pairs.
## Changes
- Return early from when there are no requested buckets or all requested
buckets are served from the DV meta cache.
- Add exact partition-bucket scan support in , using row-level filtering for
index type, partition, and bucket before deserializing matching index manifest
entries.
- Add index manifest row getters for efficient row-level filtering.
- Cover exact bucket scanning with a focused unit test, including empty
input and same-bucket different-index-type cases.
## Testing
- [INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model
for org.apache.paimon:paimon-spark-common_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @
org.apache.paimon:paimon-spark-common_${scala.binary.version}:1.5-SNAPSHOT,
/Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-common/pom.xml,
line 33, column 17
[WARNING]
[WARNING] Some problems were encountered while building the effective model
for org.apache.paimon:paimon-spark-ut_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @
org.apache.paimon:paimon-spark-ut_${scala.binary.version}:1.5-SNAPSHOT,
/Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-ut/pom.xml,
line 31, column 17
[WARNING]
[WARNING] It is highly recommended to fix these problems because they
threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support
building such malformed projects.
[WARNING]
[INFO]
------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO]
------------------------------------------------------------------------
[INFO] os.detected.name: osx
[INFO] os.detected.arch: aarch_64
[INFO] os.detected.bitness: 64
[INFO] os.detected.version: 26.3
[INFO] os.detected.version.major: 26
[INFO] os.detected.version.minor: 3
[INFO] os.detected.classifier: osx-aarch_64
[INFO]
[INFO] -------------------< org.apache.paimon:paimon-core
>--------------------
[INFO] Building Paimon : Core 1.5-SNAPSHOT
[INFO] from pom.xml
[INFO] --------------------------------[ jar
]---------------------------------
[INFO]
[INFO] --- checkstyle:2.17:check (validate) @ paimon-core ---
[INFO]
[INFO] --- spotless:2.13.0:check (spotless-check) @ paimon-core ---
[INFO] Spotless check skipped
[INFO] Spotless check skipped
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (enforce-maven-version) @ paimon-core
---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (enforce-maven) @ paimon-core ---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (ban-unsafe-snakeyaml) @ paimon-core ---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (ban-unsafe-jackson) @ paimon-core ---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (forbid-log4j-1) @ paimon-core ---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce
(forbid-direct-table-planner-dependencies) @ paimon-core ---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- directory:1.0:directory-of (directories) @ paimon-core ---
[INFO] Directory of org.apache.paimon:paimon-parent set to:
/Users/lijingsong/IdeaProjects/paimon
[INFO]
[INFO] --- remote-resources:1.7.0:process (process-resource-bundles) @
paimon-core ---
[INFO] Preparing remote bundle org.apache:apache-jar-resource-bundle:1.4
[INFO] Copying 3 resources from 1 bundle.
[INFO]
[INFO] --- resources:3.1.0:resources (default-resources) @ paimon-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO]
[INFO] --- compiler:3.8.0:compile (default-compile) @ paimon-core ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- resources:3.1.0:testResources (default-testResources) @
paimon-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 29 resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- compiler:3.8.0:testCompile (default-testCompile) @ paimon-core ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- surefire:3.0.0-M5:test (default-test) @ paimon-core ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.paimon.index.IndexFileHandlerTest
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.876
s - in org.apache.paimon.index.IndexFileHandlerTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 3.833 s
[INFO] Finished at: 2026-06-01T22:40:30+08:00
[INFO]
------------------------------------------------------------------------
- [INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model
for org.apache.paimon:paimon-spark-common_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @
org.apache.paimon:paimon-spark-common_${scala.binary.version}:1.5-SNAPSHOT,
/Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-common/pom.xml,
line 33, column 17
[WARNING]
[WARNING] Some problems were encountered while building the effective model
for org.apache.paimon:paimon-spark-ut_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @
org.apache.paimon:paimon-spark-ut_${scala.binary.version}:1.5-SNAPSHOT,
/Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-ut/pom.xml,
line 31, column 17
[WARNING]
[WARNING] It is highly recommended to fix these problems because they
threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support
building such malformed projects.
[WARNING]
[INFO]
------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO]
------------------------------------------------------------------------
[INFO] os.detected.name: osx
[INFO] os.detected.arch: aarch_64
[INFO] os.detected.bitness: 64
[INFO] os.detected.version: 26.3
[INFO] os.detected.version.major: 26
[INFO] os.detected.version.minor: 3
[INFO] os.detected.classifier: osx-aarch_64
[INFO]
[INFO] -------------------< org.apache.paimon:paimon-core
>--------------------
[INFO] Building Paimon : Core 1.5-SNAPSHOT
[INFO] from pom.xml
[INFO] --------------------------------[ jar
]---------------------------------
[INFO]
[INFO] --- checkstyle:2.17:check (validate) @ paimon-core ---
[INFO]
[INFO] --- spotless:2.13.0:check (spotless-check) @ paimon-core ---
[INFO] Spotless check skipped
[INFO] Spotless check skipped
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (enforce-maven-version) @ paimon-core
---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (enforce-maven) @ paimon-core ---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (ban-unsafe-snakeyaml) @ paimon-core ---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (ban-unsafe-jackson) @ paimon-core ---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (forbid-log4j-1) @ paimon-core ---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce
(forbid-direct-table-planner-dependencies) @ paimon-core ---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- directory:1.0:directory-of (directories) @ paimon-core ---
[INFO] Directory of org.apache.paimon:paimon-parent set to:
/Users/lijingsong/IdeaProjects/paimon
[INFO]
[INFO] --- remote-resources:1.7.0:process (process-resource-bundles) @
paimon-core ---
[INFO] Preparing remote bundle org.apache:apache-jar-resource-bundle:1.4
[INFO] Copying 3 resources from 1 bundle.
[INFO]
[INFO] --- resources:3.1.0:resources (default-resources) @ paimon-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO]
[INFO] --- compiler:3.8.0:compile (default-compile) @ paimon-core ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- resources:3.1.0:testResources (default-testResources) @
paimon-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 29 resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- compiler:3.8.0:testCompile (default-testCompile) @ paimon-core ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- surefire:3.0.0-M5:test (default-test) @ paimon-core ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.paimon.table.PrimaryKeySimpleTableTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.95
s - in org.apache.paimon.table.PrimaryKeySimpleTableTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 5.424 s
[INFO] Finished at: 2026-06-01T22:40:36+08:00
[INFO]
------------------------------------------------------------------------
- [INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model
for org.apache.paimon:paimon-spark-common_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @
org.apache.paimon:paimon-spark-common_${scala.binary.version}:1.5-SNAPSHOT,
/Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-common/pom.xml,
line 33, column 17
[WARNING]
[WARNING] Some problems were encountered while building the effective model
for org.apache.paimon:paimon-spark-ut_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @
org.apache.paimon:paimon-spark-ut_${scala.binary.version}:1.5-SNAPSHOT,
/Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-ut/pom.xml,
line 31, column 17
[WARNING]
[WARNING] Some problems were encountered while building the effective model
for org.apache.paimon:paimon-spark3-common_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @
org.apache.paimon:paimon-spark3-common_${scala.binary.version}:1.5-SNAPSHOT,
/Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark3-common/pom.xml,
line 33, column 17
[WARNING]
[WARNING] Some problems were encountered while building the effective model
for org.apache.paimon:paimon-spark-3.5_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @
org.apache.paimon:paimon-spark-3.5_${scala.binary.version}:1.5-SNAPSHOT,
/Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-3.5/pom.xml,
line 31, column 17
[WARNING]
[WARNING] Some problems were encountered while building the effective model
for org.apache.paimon:paimon-spark-3.4_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @
org.apache.paimon:paimon-spark-3.4_${scala.binary.version}:1.5-SNAPSHOT,
/Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-3.4/pom.xml,
line 31, column 17
[WARNING]
[WARNING] Some problems were encountered while building the effective model
for org.apache.paimon:paimon-spark-3.3_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @
org.apache.paimon:paimon-spark-3.3_${scala.binary.version}:1.5-SNAPSHOT,
/Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-3.3/pom.xml,
line 31, column 17
[WARNING]
[WARNING] Some problems were encountered while building the effective model
for org.apache.paimon:paimon-spark-3.2_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @
org.apache.paimon:paimon-spark-3.2_${scala.binary.version}:1.5-SNAPSHOT,
/Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-3.2/pom.xml,
line 31, column 17
[WARNING]
[WARNING] It is highly recommended to fix these problems because they
threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support
building such malformed projects.
[WARNING]
[INFO]
------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO]
------------------------------------------------------------------------
[INFO] os.detected.name: osx
[INFO] os.detected.arch: aarch_64
[INFO] os.detected.bitness: 64
[INFO] os.detected.version: 26.3
[INFO] os.detected.version.major: 26
[INFO] os.detected.version.minor: 3
[INFO] os.detected.classifier: osx-aarch_64
[INFO]
[INFO] -------------------< org.apache.paimon:paimon-core
>--------------------
[INFO] Building Paimon : Core 1.5-SNAPSHOT
[INFO] from pom.xml
[INFO] --------------------------------[ jar
]---------------------------------
[INFO]
[INFO] --- checkstyle:2.17:check (validate) @ paimon-core ---
[INFO]
[INFO] --- spotless:2.13.0:check (spotless-check) @ paimon-core ---
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (enforce-maven-version) @ paimon-core
---
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (enforce-maven) @ paimon-core ---
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (ban-unsafe-snakeyaml) @ paimon-core ---
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (ban-unsafe-jackson) @ paimon-core ---
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (forbid-log4j-1) @ paimon-core ---
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce
(forbid-direct-table-planner-dependencies) @ paimon-core ---
[INFO]
[INFO] --- directory:1.0:directory-of (directories) @ paimon-core ---
[INFO] Directory of org.apache.paimon:paimon-parent set to:
/Users/lijingsong/IdeaProjects/paimon
[INFO]
[INFO] --- remote-resources:1.7.0:process (process-resource-bundles) @
paimon-core ---
[INFO] Preparing remote bundle org.apache:apache-jar-resource-bundle:1.4
[INFO] Copying 3 resources from 1 bundle.
[INFO]
[INFO] --- resources:3.1.0:resources (default-resources) @ paimon-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO]
[INFO] --- compiler:3.8.0:compile (default-compile) @ paimon-core ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 15.049 s
[INFO] Finished at: 2026-06-01T22:40:52+08:00
[INFO]
------------------------------------------------------------------------
-
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
