Gabriel39 opened a new pull request, #69651:
URL: https://github.com/apache/doris/pull/69651
### What problem does this PR solve?
Related PR: #69650 (reader regression coverage)
With `nodes_discovery=true`, ES metadata loading follows
`EsMetadataFetcher.fetchShards -> EsConnectorRestClient.getHttpNodes ->
EsNodeInfo -> EsMajorVersion.parse`. The parser accepts only majors through 8,
so a 9.x node makes discovery fail with an unsupported-version exception. The
configured-seed path with discovery disabled does not parse node versions and
is not affected in the same way.
Add the explicit ES 9 major, comparisons and latest-supported bound while
continuing to reject unvalidated future majors with a clear diagnostic. Add an
HTTP unit test containing mixed 8.19/9.0 nodes, IPv6 routing and a node without
HTTP, plus an opt-in live compatibility suite rather than relying on version
parsing alone.
The live suite creates isolated temporary indices and checks the production
connector's `_nodes/http`, `_mapping`, `_search_shards`, index listing and
metadata fetching with discovery enabled/disabled. It also verifies
authenticated source/docvalue scroll requests, missing values, pagination,
scroll cleanup and invalid-credential failures. Enable it with
`DORIS_ES9_TEST_URL`, `DORIS_ES9_TEST_USER` and `DORIS_ES9_TEST_PASSWORD`.
### Release note
Allow Elasticsearch 9.x nodes during ES catalog node discovery.
### Check List (For Author)
- Test
- [x] Version parsing and mixed-node tests reproduce the unsupported-9.x
failure before the change.
- [x] All 124 ES connector tests pass with live authenticated
Elasticsearch 9.0.0 and 9.5.5, including the three opt-in compatibility tests
on each version.
- [x] Default unit-test execution passes with the live suite disabled
when no endpoint is configured.
- [x] FE Checkstyle for the ES module and its reactor dependencies.
- [x] The BE ES reader also reads the authenticated three-row fixture
through source/docvalue scans with batch size 1 on both versions, using the
reader coverage change in the related ES regression PR.
- [ ] Full Doris SQL end-to-end validation against ES 9; the live tests
above validate connector metadata and HTTP reader compatibility.
- Behavior changed:
- [x] Yes: discovery accepts ES 9 nodes; later unverified majors remain
rejected.
- Does this need documentation?
- [x] No new catalog properties or configuration are introduced.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
