Standing-Man opened a new issue, #66675: URL: https://github.com/apache/doris/issues/66675
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Description The table schema API: ```text GET /api/{db}/{table}/_schema GET /api/{catalog}/{db}/{table}/_schema ``` currently exposes only the top-level primitive type through the `type` field. The legacy `precision` and `scale` fields are provided only when the column itself is a decimal type. For example: ```sql CREATE TABLE doris_types ( types_id INT, c_decimal ARRAY<DECIMAL(18, 4)> ) UNIQUE KEY(types_id) DISTRIBUTED BY HASH(types_id) BUCKETS 1 PROPERTIES ( "replication_num" = "1" ); ``` The schema response identifies `c_decimal` only as: ```json { "name": "c_decimal", "type": "ARRAY" } ``` Consumers cannot discover that the array element is `DECIMAL(18, 4)`, including its precision and scale. The same limitation applies to nested `ARRAY`, `MAP`, and `STRUCT` combinations. This affects external connectors that use the Doris schema API for data conversion. For example, RisingWave's Doris sink attempted to retrieve decimal precision and scale from the top-level `ARRAY` type and panicked because this information was unavailable: https://github.com/risingwavelabs/risingwave/issues/16176 Although connectors should handle missing metadata without panicking, Doris should expose enough schema information for clients to interpret nested column types reliably. ### Solution Add two backward-compatible fields to every column returned by the table schema API: - `type_sql`: Complete SQL representation of the column type. - `type_desc`: Recursively structured type metadata. For `ARRAY<DECIMAL(18, 4)>`, the response would include: ```json { "name": "c_decimal", "type": "ARRAY", "type_sql": "array<decimalv3(18,4)>", "type_desc": { "kind": "ARRAY", "sql": "array<decimalv3(18,4)>", "contains_null": true, "element": { "kind": "DECIMAL64", "sql": "decimalv3(18,4)", "precision": 18, "scale": 4 } } } ``` The recursive representation should support: - `ARRAY`: element type and element nullability. - `MAP`: key/value types and their nullability. - `STRUCT`: ordered fields, field types, and field nullability. - Decimal types: precision and scale. - `CHAR` and `VARCHAR`: length. - `DATETIMEV2`, `TIMEV2`, and `TIMESTAMPTZ`: scale. - Primitive types: type kind and SQL representation. Fields that do not apply to a type should be omitted. Existing fields such as `type`, `precision`, and `scale` should remain unchanged for backward compatibility. The new metadata should be included for both base-table columns and materialized-index columns. Add two backward-compatible fields to every column returned by the table schema API: - `type_sql`: Complete SQL representation of the column type. - `type_desc`: Recursively structured type metadata. For `ARRAY<DECIMAL(18, 4)>`, the response would include: ```json { "name": "c_decimal", "type": "ARRAY", "type_sql": "array<decimalv3(18,4)>", "type_desc": { "kind": "ARRAY", "sql": "array<decimalv3(18,4)>", "contains_null": true, "element": { "kind": "DECIMAL64", "sql": "decimalv3(18,4)", "precision": 18, "scale": 4 } } } ``` The recursive representation should support: - `ARRAY`: element type and element nullability. - `MAP`: key/value types and their nullability. - `STRUCT`: ordered fields, field types, and field nullability. - Decimal types: precision and scale. - `CHAR` and `VARCHAR`: length. - `DATETIMEV2`, `TIMEV2`, and `TIMESTAMPTZ`: scale. - Primitive types: type kind and SQL representation. Fields that do not apply to a type should be omitted. Existing fields such as `type`, `precision`, and `scale` should remain unchanged for backward compatibility. The new metadata should be included for both base-table columns and materialized-index columns. ### Acceptance criteria - Nested decimal precision and scale can be obtained without parsing type strings. - Arbitrarily nested `ARRAY`, `MAP`, and `STRUCT` types are represented recursively. - Nullability is preserved at each nested level. - Existing schema API consumers remain compatible. - Base-table and materialized-index schemas use the same representation. ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
