morningman opened a new pull request, #67487:
URL: https://github.com/apache/doris/pull/67487
### What problem does this PR solve?
Issue Number: close #67368
Related PR: #67381 (sibling fix in the same tracking series), #62259
Problem Summary:
A `UNIQUE KEY` point query that qualifies for the short-circuit path
returned no Flight endpoint over Arrow Flight SQL. The client failed with
```
fetch arrow flight schema failed, no FlightSqlEndpointsLocations
```
and the row was silently dropped. The identical query with
`SET_VAR(enable_short_circuit_query=false)` returned it on the same connection.
**Root cause** — the short circuit produces no Arrow result at either end,
and nothing prevented an Arrow Flight connection from planning one:
* It executes on `PointQueryExecutor`, not a `Coordinator`, and
`Coordinator`/`NereidsCoordinator` are the only places that register a
`FlightSqlEndpointsLocation`. `StmtExecutor.executeAndSendResult` then returns
early through its Arrow Flight branch with nothing registered and without ever
calling `getNext()`, so `GetFlightInfo` found an empty endpoint list.
* The BE cannot be pointed at either. `tablet_fetch_data` serializes with
`VMysqlResultWriter` into `PTabletKeyLookupResponse.row_batch` and runs no
fragment, so the `ArrowFlightResultBlockBuffer` that
`fetch_arrow_flight_schema` looks up by finst id never exists.
`LogicalResultSinkToShortCircuitPointQuery` did not look at the connect
type, and `enable_short_circuit_query` defaults to `true`, so every ADBC /
Arrow Flight JDBC point query on a MoW + light-schema-change +
`store_row_column` table hit this. Prepared statements go through the same
`executeQueryStatement` and failed identically.
**Fix** — keep Arrow Flight SQL on the normal execution path.
This has to be decided at plan time rather than when picking the executor:
`OlapScanNode.computeTabletInfo` and several rewrite/property rules
(`ChildOutputPropertyDeriver`, `ShuffleKeyPruner`, `NestedColumnPruning`,
`PruneOlapScanPartition`) read `StatementContext.isShortCircuitQuery()` while
the plan is being built, so flipping the flag later would run a coordinator
over a plan shaped for a different execution mode. MySQL connections keep the
short circuit unchanged.
Returning the point-query result from the FE instead was considered and
rejected for now: `FlightSqlChannel.addResult` builds varchar vectors only, so
every column would come back as `Utf8`, inconsistent with the normal Flight
path. Full support (Arrow serialization in the BE lookup RPC plus a result
buffer to hand out an endpoint) is a larger change and out of scope here.
Also refreshes a now-stale comment in `StmtExecutor` that said point queries
reach the Arrow Flight deferral gate.
### Release note
Fix Arrow Flight SQL returning no endpoint (`no
FlightSqlEndpointsLocations`) for a point query that hits the short-circuit
path.
### Check List (For Author)
- Test
- [x] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
`regression-test/suites/arrow_flight_sql_p0/test_point_query_over_arrow_flight.groovy`
uses the table from the issue and holds both a MySQL and an Arrow Flight
connection. It asserts the MySQL explain still contains `SHORT-CIRCUIT` (so a
table that stopped qualifying fails loudly instead of making the rest pass for
the wrong reason), that the Flight explain does not, that both protocols return
the same row, and that the numeric columns stay numeric over Flight.
`ShortCircuitPointQueryTest#testArrowFlightSqlConnectionDoesNotUseShortCircuit`
covers the rule itself and re-asserts that the same statement still short
circuits on a MySQL connection.
Verified on a local single-FE + single-BE cluster, all three runs on the
same cluster:
| Run | FE | Result |
|---|---|---|
| 1 | with the fix | `All suites success`, `Test 1 suites, failed 0` |
| 2 | guard temporarily disabled | **failed** — `the point query must not
short circuit on an arrow flight connection` |
| 3 | fix restored | `All suites success`, `Test 1 suites, failed 0` |
Run 2 is the negative control; the explain it captured is the bug itself,
with a Flight result sink and the short circuit in one plan:
```
VRESULT SINK
ARROW_FLIGHT_PROTOCOL
0:VOlapScanNode(28)
...
SHORT-CIRCUIT
```
FE UT: `Tests run: 8, Failures: 0, Errors: 0` for
`ShortCircuitPointQueryTest`.
- Behavior changed:
- [ ] No.
- [x] Yes. A point query over Arrow Flight SQL is now planned on the
normal execution path instead of the short circuit, so it returns its row
instead of failing. It gives up the point-query latency benefit on that
protocol; MySQL connections are unaffected.
- Does this need documentation?
- [x] No.
- [ ] Yes.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_017omcvWDsxEc9AyU83aBZ2g
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
