DaZuiZui commented on issue #18846:
URL: https://github.com/apache/iotdb/issues/18846#issuecomment-6160193918

   # Corrected functional definition: result column mapping for Table-Model 
`PreparedStatement`
   
   > It adds no grammar and changes no parsing rule. What follows is the 
**result-column-mapping semantics** of already-existing syntax on the JDBC 
client, i.e. the behaviour contract the fix satisfies.
   
   ## 1. Syntax definition (affected surface)
   
   The mapping semantics apply to every Table Model query using `?` 
placeholders, in particular the three shapes below:
   
   ```sql
   -- (a) projection order differs from the materialized order
   SELECT b, a FROM t WHERE a > ?;
   
   -- (b) repeated projection / aliases sharing one physical column
   SELECT a AS x, a AS y FROM t WHERE a > ?;
   
   -- (c) parameter in WHERE, time column in the projection
   SELECT time, s1 FROM metrics WHERE device = ? AND s1 > ?;
   ```
   
   Key points:
   
   - The projection list fixes the **logical result-column order**, which may 
differ from the **physical column order** of the server's TsBlock.
   - The correspondence between the two is determined solely by the mapping the 
server sends; the client adopts it and never derives it on its own.
   - One physical column may be referenced by several logical columns (case 
(b)); the server may therefore send repeated entries.
   
   ## 2. Functional description
   
   ### 2.1 Components and their meanings
   
   | Component | Source | Meaning |
   | --- | --- | --- |
   | `columns` | `resp.getColumns()` | Logical result column names; their order 
is the result-set column order |
   | `columnIndex2TsBlockColumnIndexList` | `resp` | **Authoritative mapping**: 
logical column ordinal → physical TsBlock column index |
   | `columnNameIndexMap` | `resp.columnNameIndexMap` | Not consumed by the 
client. `IoTDBRpcDataSet` accepts it as the `columnNameIndex` parameter 
(`IoTDBRpcDataSet.java:99`) and never reads it again. The name → ordinal lookup 
actually used is `columnOrdinalMap` (`:61`, *"used because the server returns 
deduplicated columns"*), built internally from `columnNameList` (`:181-184`) 
with first-occurrence-wins. Passing this field alone therefore cannot preserve 
the mapping — which is exactly the point of §4 |
   | `dataTypeList` | `resp.getDataTypeList()` | Type declaration **per logical 
column, not per physical column**. It is indexed by logical ordinal — 
`IoTDBRpcDataSet.java:173` reads `columnTypeList.get(i)` where `i` walks 
`columnNameList` — and it is explicitly **not deduplicated** (`:59`). Its 
length equals `columns.size()`, which is *not* the physical block width |
   | `tableModel` | `resp.isSetTableModel() && resp.isTableModel()` | 
Table-model flag. Note it is currently **accepted and discarded**: 
`IoTDBRpcDataSet.java:112` declares the parameter and never stores or reads it. 
Forwarding it is still required for parity with `IoTDBStatement` |
   | `ignoreTimeStamp` | `resp` | Whether the tree-model time column is hidden. 
Per `IoTDBRpcDataSet.java:114`, *"only used for tree model, table model this 
field will always be true"* — so no `-1` entry is prepended |
   | `charset` | the connection | Charset used by the `getBytes(...)` getters 
(`IoTDBJDBCResultSet.java:391,412`) |
   
   ### 2.2 Rules
   
   1. **Adopt the mapping as sent.** The client uses the server-provided 
`columnIndex2TsBlockColumnIndexList` verbatim and never rebuilds it as an 
identity mapping.
   2. **Duplicates are allowed.** `m[i]` values may repeat; a repeated logical 
column returns the value of the physical column it refers to.
   3. **Physical block width** is `max(m) + 1` 
(`IoTDBRpcDataSet.java:163-166`); the client sizes its column-type arrays from 
that, **not** from the number of logical columns.
   4. **Index-based and name-based reads share the same mapping** and return 
identical values.
   5. **Duplicate logical column names resolve to the first occurrence** 
(`:181-184`).
   6. Only when the mapping is absent does the client fall back to the existing 
derivation (tree model: `-1` placeholder for the time column plus a shift; 
identity otherwise).
   7. `tableModel` comes from `resp.isSetTableModel() && resp.isTableModel()` 
and is never hard-coded.
   
   ### 2.3 Types
   
   `dataTypeList` carries one of the `TSDataType` names. The full enum is 
`BOOLEAN, INT32, INT64, FLOAT, DOUBLE, TEXT, STRING, VECTOR, UNKNOWN, 
TIMESTAMP, DATE, BLOB, OBJECT`. Two levels stay apart:
   
   **a. What can appear as a mapped value column** — `BOOLEAN, INT32, INT64, 
FLOAT, DOUBLE, TEXT, STRING, DATE, BLOB, OBJECT`. (`VECTOR` and `UNKNOWN` are 
enum members that are not produced on this path.)
   
   **b. `TIMESTAMP`** — in the table model there is no time column, so it never 
appears as a mapped value column. It arises only from the `-1` time-column 
placeholder in the tree model.
   
   Behaviour per type on the getters:
   
   - **`OBJECT`** — non-string getters throw `SQLException` with 
`OBJECT_ERR_MSG` (`IoTDBJDBCResultSet.java:74,313,335,385,406`); `getString` 
works via `BytesUtils.parseObjectByteArrayToString` 
(`IoTDBRpcDataSet.java:562-564`).
   - **`DATE`** — materialized as `INT32`; `getDate` is `getInt` + 
`DateUtils.parseIntToLocalDate` (`:597-601`), and `getString` formats via 
`DateUtils.formatDate` (`:568-569`).
   - **`TEXT` / `STRING`** — decoded with `TSFileConfig.STRING_CHARSET`, 
**not** with the connection charset (`:556-561`).
   - **`BLOB`** — returned as `getBinary(...).getValues()` (`:565-567`).
   - **When several logical columns share one physical column**, 
`dataTypeForTsBlockColumn` is written once per logical column (`:179`), so a 
later logical column overwrites the type recorded for that physical column. The 
current code does not detect or reject such a conflict.
   - **Index constraint** — see §4: out-of-range indices do **not** currently 
raise `SQLException`.
   
   ## 3. Examples
   
   The server sends `columns = ["b","a"]`, `dataTypeList = ["INT32","INT32"]`, 
`columnIndex2TsBlockColumnIndexList = [1,0]`, and the physical TsBlock holds 
two INT32 columns with values `10` and `20`:
   
   ```java
   PreparedStatement ps = conn.prepareStatement("SELECT b, a FROM t WHERE a > 
?");
   ps.setInt(1, 0);
   ResultSet rs = ps.executeQuery();
   rs.next();
   rs.getInt(1);        // expected 20 (logical b → physical 1)
   rs.getInt("a");      // expected 10 (logical a → physical 0)
   ```
   
   | `columns` | mapping | physical values | expected `[col 1, col 2]` | 
expected by name |
   | --- | --- | --- | --- | --- |
   | `["b","a"]` | `[1,0]` | `[10, 20]` | `[20, 10]` | `b=20, a=10` |
   | `["x","y"]` | `[0,0]` | `[10]` | `[10, 10]` | `x=10, y=10` (must not 
throw) |
   | `["a","b"]` | `[0,1]` | `[10, 20]` | `[10, 20]` | identity case, 
regression control |
   
   Current behaviour for the same three rows: the identity mapping is 
substituted, so row 1 silently returns `[10, 20]` instead of `[20, 10]`, and 
row 2 resolves column 2 to `list.get(1)` = `1` and then calls 
`curTsBlock.getColumn(1)` on a one-column block, raising 
`ArrayIndexOutOfBoundsException`.
   
   **Server reachability.** This is not a synthetic-only fixture. 
`DatasetHeader.setTableColumnToTsBlockIndexMap` (`DatasetHeader.java:100-107`) 
builds the mapping by looking up each output symbol's position in the *child* 
output symbol list, so a projection order that differs from the scan order 
yields a non-identity mapping, and a symbol used twice yields duplicate entries.
   
   ## 4. Current divergence and fix
   
   `IoTDBTablePreparedStatement.processQueryResult()` 
(`iotdb-client/jdbc/src/main/java/org/apache/iotdb/jdbc/IoTDBTablePreparedStatement.java:215-232`)
 calls the 14-argument `IoTDBJDBCResultSet` constructor 
(`IoTDBJDBCResultSet.java:139-181`), which hard-codes `false` for `tableModel` 
(`:173`) and substitutes `IntStream.range(0, columnNameList.size())` for the 
mapping (`:174`). That constructor also never assigns `this.charset`, so the 
field keeps its initializer `TSFileConfig.STRING_CHARSET` (`:86`). Ordinary 
`IoTDBStatement` (`:541-558`) calls the full constructor and forwards all 
three, so the same response yields different results on the two paths.
   
   **Fix.** In `IoTDBTablePreparedStatement.java`, replace the result-set 
construction at `:218-232` with the full constructor, adding after `zoneId`:
   
   ```java
   charset,
   resp.isSetTableModel() && resp.isTableModel(),
   new ArrayList<>(resp.getColumnIndex2TsBlockColumnIndexList())
   ```
   
   `charset` is already in scope — inherited from `IoTDBStatement` and assigned 
by this class's own constructor (`:92-94`) — and the call shape mirrors 
`IoTDBStatement.java:557-558`. The mapping is passed as a defensive copy 
because `IoTDBRpcDataSet` prepends `-1` in place when `!ignoreTimeStamp` 
(`:167-170`) and the thrift getter hands back the response's internal list 
(`TSExecuteStatementResp.java:815-817`).
   
   ### On the index / empty-slot requirement
   
   This fix resolves the issue in full, and it does so without any separate 
decision on bounds checking. My earlier version demanded that "an out-of-range 
or empty slot raise `SQLException`, not `ArrayIndexOutOfBoundsException`", and 
that requirement is hereby dropped rather than deferred. The reason it is 
unnecessary is the invariant of the mapping itself:
   
   - `getTsBlockColumnIndexForColumnIndex` (`IoTDBRpcDataSet.java:641-643`) is 
a bare `columnIndex2TsBlockColumnIndexList.get(columnIndex - 1)`.
   - The physical block width is `max(m) + 1`, so every valid logical ordinal 
`i` satisfies `0 <= m[i] <= width - 1`. Once the mapping is adopted as sent, 
`curTsBlock.getColumn(m[i])` cannot be out of range.
   - The `ArrayIndexOutOfBoundsException` reported in §3 is an artifact of the 
identity mapping being applied to a one-column block. It disappears together 
with the mapping itself.
   
   Bounds validation for out-of-range ordinals, unknown column names, and 
`findColumn` (`IoTDBJDBCResultSet.java:537-543,617-619,633-637`) is a 
robustness improvement that stands on its own and does not gate this fix. 
`findColumnNameByIndex` (`:621-629`) already shows the pattern.
   
   The 14-argument `IoTDBJDBCResultSet` constructor 
(`IoTDBJDBCResultSet.java:139-181`) has no remaining callers after this change.
   
   ## 5. Acceptance
   
   1. All three rows in §3 return the expected values, identical to the 
`Statement` path for the same response.
   2. No `ArrayIndexOutOfBoundsException`.
   3. Every case asserts `getXxx(int)`, `getXxx(String)` **and the actual 
values** — asserting metadata alone does not pass, because the metadata itself 
is what is wrong.
   4. Both shapes are covered: reordered columns, and repeated 
projections/aliases.
   5. Test carriers: the mocked-RPC unit cases extend 
`iotdb-client/jdbc/src/test/java/org/apache/iotdb/jdbc/IoTDBTablePreparedStatementTest.java`,
 and the real-server cases extend 
`integration-test/src/test/java/org/apache/iotdb/relational/it/db/it/IoTDBTablePreparedStatementJDBCIT.java`
 and `IoTDBTablePreparedStatementIT.java`, which is what demonstrates 
reachability.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to