CrownChu opened a new pull request, #8000:
URL: https://github.com/apache/paimon/pull/8000
Summary
- Inject shared ExecutorService from ESIndexGlobalIndexerFactory for
DiskBBQ parallel cluster search, replacing per-reader thread pool creation
- Add volatile closed flag with checkNotClosed() for concurrent close
safety (fixes race condition in reader lifecycle)
- Implement all scalar/keyword/text filter visitor methods (visitEqual,
visitLessThan, visitStartsWith, visitLike, visitIsNull, etc.) via ESLib
IndexFilter / ScalarPredicate API
- Change eslib dependency groupId from org.elasticsearch to
io.github.crownchu, version from 1.0.0-SNAPSHOT to 1.0.0
- Add GitHub-hosted Maven repository for public CI dependency resolution
Details
Parallel Search
Search thread pool lifecycle is owned by the factory layer
(ESIndexGlobalIndexerFactory), lazily initialized and shared across all
readers. Configurable via global-index.es-index.read-search-threads:
- -1 (default): auto = CPU/2, min 2
- 0: disable parallel search (serial only)
- >0: use specified thread count
The executor is injected through the full chain: Factory → Indexer →
Reader → ESIndexSearcher → Lucene Codec (via SearchExecutorHolder ThreadLocal
bridge for Lucene SPI no-arg constructor constraint).
Scalar Filter
ESIndexGlobalIndexReader now implements all GlobalIndexReader visitor
methods, dispatching to ESLib's unified filter API:
- Numeric comparisons → ScalarPredicate.eq/lt/lte/gt/gte/in/notIn
- Text matching → IndexFilter.TextFilter with TERM/PREFIX/WILDCARD ops
- Null checks → IndexFilter.exists() / notExists()
Dependency Publishing
ESLib jars are published to a GitHub raw repository
(CrownChu/es-paimon-lib-releases) with full Maven metadata and checksums.
paimon-eslib/pom.xml declares the repository so CI can resolve dependencies
without manual local installation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
