Doris-Breakwater commented on issue #67365:
URL: https://github.com/apache/doris/issues/67365#issuecomment-5490548672
Breakwater-GitHub-Analysis-Slot: slot_ba4943dd2476
## Initial assessment
This is a credible FE/JDBC metadata error-contract issue, but the
cross-version description needs one important correction before implementation.
The issue is currently unlabelled and unassigned; it should be routed to the FE
external-catalog/JDBC maintainers.
### Code-verified facts
- **`branch-4.0` (`8a99617`)**: the reported NPE chain is valid.
`JdbcExternalTable.initSchema()` returns `Optional.empty()` for null/empty
columns; `ExternalSchemaCache` stores that `Optional` value;
`ExternalTable.getFullSchema()` converts it to `null`; and
`LogicalCatalogRelation.computeOutput()` calls `.stream()` without a null
check. The negative result remains cached subject to the schema-cache
expiry/refresh policy.
- **`branch-4.1` (`6f4c6a4`)**: the reported empty-schema NPE/negative-cache
behavior is **not** the normal behavior at this SHA. This commit already
contains the `branch-4.1` version of #66633 (`9a7174edb22`).
`DefaultExternalMetaCache.loadSchemaCacheValue()` converts
`ExternalCatalog.getSchema(...)=Optional.empty()` into `CacheException`.
Consequently, null/empty JDBC columns fail during the cache load, the failure
is not retained as a negative entry, and repeated statements can reload remote
metadata. With preload enabled the exception is raised by the preload
`getBaseSchema()` call; without preload it is raised by the later schema
access. For this condition, the value does not normally reach
`LogicalCatalogRelation.getBaseSchema().stream()` as `null`.
- **`master` (`feb9e04`)**: a missing connector table handle follows the
same generic `CacheException`/uncached-failure path. A valid handle with an
empty `ConnectorTableSchema` is different: `ConnectorTableSchema` normalizes
null columns to an empty list, `PluginDrivenExternalTable` creates a present
cache value, and `SchemaCacheValue.validateSchema()` does not reject an empty
schema. `computeOutput()` then streams an empty list; the report does not yet
demonstrate the resulting user-visible failure or incorrect plan.
- `PreloadExternalMetadata` only calls the table metadata APIs. It does not
invalidate the schema cache or catch/suppress their exceptions. The
repeated-load behavior comes from the failed cache load not producing a
cacheable value, not from preload invalidating the entry.
### Evidence still needed
1. Full FE exception/stack trace for each claimed SHA, with preload enabled
and disabled. In particular, an NPE observed on `6f4c6a4` would indicate
another cache route or race and needs a complete stack trace.
2. A deterministic reproducer or unit-test fixture identifying which
condition is exercised: legacy `listColumns()` returns null/empty, connector
`getTableHandle()` returns empty, or a valid handle returns zero columns.
3. JDBC database/driver name and version, metadata permissions, and whether
the remote table was concurrently dropped/recreated.
4. Remote-call counters or debug logs for repeated statements, separating
table-existence/handle resolution from column-metadata reads, plus the
effective schema-cache TTL/capacity settings and any explicit `REFRESH`
operations.
5. For the `master` zero-column case, the actual analysis result/stack trace
and expected contract. The current code proves that an empty schema is
accepted, but not by itself that it produces the same failure as a missing
handle.
### Recommended next steps
1. Correct the issue's version matrix: `branch-4.0` has the nullable-schema
NPE and cached absence; `branch-4.1` and `master` have generic uncached loader
failures for missing schema/handle after #66633.
2. Add failing tests first for the three distinct cases above. Assert
exception type/message, preload on/off parity, and remote-call count across
repeated statements. Keep a normal non-empty schema control case.
3. Fix the branches separately:
- On `branch-4.0`, preserve the bounded cached absence but convert it at
the JDBC table schema-consumption boundary into an actionable
`JdbcClientException` containing catalog and remote database/table names; no
required schema API should return `null`.
- On `branch-4.1` and `master`, an override that only checks
`getFullSchema()` is insufficient because the unified loader throws before
returning. Introduce a non-null, bounded negative result (or a dedicated
failure cache/result type) with expiry and explicit-refresh invalidation, then
translate it to the contextual JDBC error at the consumption boundary.
- On `master`, explicitly define whether a normal JDBC table may ever
have zero columns. If not, reject that state with JDBC table context before
publishing a normal schema cache value; avoid applying a blanket rule to
connector paths that intentionally use empty schemas unless their contracts are
checked.
4. Keep `PreloadExternalMetadata` generic unless a separate preload-specific
defect is reproduced; the schema provider/cache boundary should own consistent
error semantics.
No code changes were made during this triage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
