morningman commented on PR #46646:
URL: https://github.com/apache/doris/pull/46646#issuecomment-2579831331
This pull request introduces several enhancements and new features related
to Elasticsearch (ES) integration in the project. The most significant changes
include the addition of `EsNodeDiscovery` for automatic node detection, updates
to `EsExternalCatalog` to manage available nodes, and modifications to the
`CatalogMgr` and related classes to support these new features.
### Enhancements to Elasticsearch Integration:
* **Automatic Node Discovery:**
- Added `EsNodeDiscovery` class for periodic detection of available ES
nodes.
(`fe/fe-core/src/main/java/org/apache/doris/datasource/es/EsNodeDiscovery.java`)
- Integrated `EsNodeDiscovery` with `InternalCatalog` to manage ES node
information. (`fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java`,
`fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java`)
[[1]](diffhunk://#diff-bbd4eef320c67a1b2702d6cabf12b794dd8efb6b1bc61e9b70ebc639f4a0ece0R4595-R4598)
[[2]](diffhunk://#diff-f6ca4334277645de268d5ba725dc96ea03e452ca10d5125ae9bf7d4a4e50cfaaR232-R233)
* **Catalog Management:**
- Updated `CatalogMgr` to register and deregister ES catalogs with
`EsNodeDiscovery`.
(`fe/fe-core/src/main/java/org/apache/doris/datasource/CatalogMgr.java`)
[[1]](diffhunk://#diff-740530c8218ff618fa595540a1411a150359f6a2f77c04b01ea1f0f4ca5a85aeR138-R140)
[[2]](diffhunk://#diff-740530c8218ff618fa595540a1411a150359f6a2f77c04b01ea1f0f4ca5a85aeR160-R162)
* **Elasticsearch Catalog Enhancements:**
- Modified `EsExternalCatalog` to track and update available nodes
information.
(`fe/fe-core/src/main/java/org/apache/doris/datasource/es/EsExternalCatalog.java`)
[[1]](diffhunk://#diff-4e4631603dee3529f9b3027762745e84670c7ee901b53b98cbacb5e2cd4323e2R50-R64)
[[2]](diffhunk://#diff-4e4631603dee3529f9b3027762745e84670c7ee901b53b98cbacb5e2cd4323e2R173-R213)
- Enhanced `EsRestClient` to fetch a list of HTTP nodes.
(`fe/fe-core/src/main/java/org/apache/doris/datasource/es/EsRestClient.java`)
* **ES Table and Search Context:**
- Updated `EsTable` to include available nodes information.
(`fe/fe-core/src/main/java/org/apache/doris/catalog/EsTable.java`)
- Modified `SearchContext` to utilize available nodes information during
query execution.
(`fe/fe-core/src/main/java/org/apache/doris/datasource/es/SearchContext.java`,
`fe/fe-core/src/main/java/org/apache/doris/datasource/es/PartitionPhase.java`)
[[1]](diffhunk://#diff-db9fb30f868b78eb1f3e6e3fb69609cb124e9b89fdb57f32884170bfa082e754R94-R104)
[[2]](diffhunk://#diff-905adc58139551e2f8684af1afc2e0343e459565fd9217238ad697fc81184d4bL39-R44)
### Dependency Updates:
* **Test Dependency:**
- Added `mockito-core` as a test dependency in the `pom.xml` file.
(`fe/fe-core/pom.xml`)
### Code Quality:
* **Checkstyle Suppression:**
- Added suppression for static import checks in `EsExternalCatalogTest`.
(`fe/check/checkstyle/suppressions.xml`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
