DaZuiZui commented on issue #18846:
URL: https://github.com/apache/iotdb/issues/18846#issuecomment-6142620847

   # Corrected functional definition: result column mapping for Table-Model 
`PreparedStatement`
   
   > This supersedes the definition in my earlier comment on this issue. That 
version stated the component meanings incorrectly in two places and mixed two 
different levels in the type list; corrections are marked **⚠ corrected** 
below. It also paired an index requirement with a fix that cannot deliver it — 
§4.
   > It adds no grammar and changes no parsing rule. What follows is the 
**result-column-mapping semantics** of already-existing syntax on the JDBC 
client, i.e. the behaviour contract the fix must satisfy.
   
   ## 1. Syntax definition (affected surface)
   
   The mapping semantics apply to every Table Model query using `?` 
placeholders, in particular the three shapes below:
   
   ```sql
   -- (a) projection order differs from the materialized order
   SELECT b, a FROM t WHERE a > ?;
   
   -- (b) repeated projection / aliases sharing one physical column
   SELECT a AS x, a AS y FROM t WHERE a > ?;
   
   -- (c) parameter in WHERE, time column in the projection
   SELECT time, s1 FROM metrics WHERE device = ? AND s1 > ?;
   ```
   
   Key points:
   
   - The projection list fixes the **logical result-column order**, which may 
differ from the **physical column order** of the server's TsBlock.
   - The correspondence between the two is determined solely by the mapping the 
server sends; the client must not derive it on its own.
   - One physical column may be referenced by several logical columns (case 
(b)); the server may therefore send repeated entries.
   
   ## 2. Functional description
   
   ### 2.1 Components and their meanings
   
   | Component | Source | Meaning |
   | --- | --- | --- |
   | `columns` | `resp.getColumns()` | Logical result column names; their order 
is the result-set column order |
   | `columnIndex2TsBlockColumnIndexList` | `resp` | **Authoritative mapping**: 
logical column ordinal → physical TsBlock column index |
   | `columnNameIndexMap` | `resp.columnNameIndexMap` | ⚠ **Not consumed by the 
client.** `IoTDBRpcDataSet` accepts it as the `columnNameIndex` parameter 
(`IoTDBRpcDataSet.java:99`) and never reads it again. The name → ordinal lookup 
actually used is `columnOrdinalMap` (`:61`, *"used because the server returns 
deduplicated columns"*), built internally from `columnNameList` (`:181-184`) 
with first-occurrence-wins. Passing this field alone therefore cannot preserve 
the mapping — which is exactly the point of §4 |
   | `dataTypeList` | `resp.getDataTypeList()` | ⚠ Type declaration **per 
logical column, not per physical column**. It is indexed by logical ordinal — 
`IoTDBRpcDataSet.java:173` reads `columnTypeList.get(i)` where `i` walks 
`columnNameList` — and it is explicitly **not deduplicated** (`:59`). Its 
length equals `columns.size()`, which is *not* the physical block width |
   | `tableModel` | `resp.isSetTableModel() && resp.isTableModel()` | 
Table-model flag. Note it is currently **accepted and discarded**: 
`IoTDBRpcDataSet.java:112` declares the parameter and never stores or reads it. 
Forwarding it is still required for parity with `IoTDBStatement` |
   | `ignoreTimeStamp` | `resp` | Whether the tree-model time column is hidden. 
Per `IoTDBRpcDataSet.java:114`, *"only used for tree model, table model this 
field will always be true"* — so no `-1` entry is prepended |
   | `charset` | the connection | Charset used by the `getBytes(...)` getters 
(`IoTDBJDBCResultSet.java:391,412`) |
   
   ### 2.2 Rules
   
   1. **Adopt the mapping as sent.** The client uses the server-provided 
`columnIndex2TsBlockColumnIndexList` verbatim and must not rebuild it as an 
identity mapping.
   2. **Duplicates are allowed.** `m[i]` values may repeat; a repeated logical 
column must return the value of the physical column it refers to.
   3. **Physical block width** is `max(m) + 1` 
(`IoTDBRpcDataSet.java:163-166`); the client sizes its column-type arrays from 
that, **not** from the number of logical columns.
   4. **Index-based and name-based reads share the same mapping** and must 
agree.
   5. **Duplicate logical column names resolve to the first occurrence** 
(`:181-184`).
   6. Only when the mapping is absent does the client fall back to the existing 
derivation (tree model: `-1` placeholder for the time column plus a shift; 
identity otherwise).
   7. `tableModel` must come from `resp.isSetTableModel() && 
resp.isTableModel()` and must not be hard-coded.
   
   ### 2.3 Types
   
   `dataTypeList` carries one of the `TSDataType` names. The full enum is 
`BOOLEAN, INT32, INT64, FLOAT, DOUBLE, TEXT, STRING, VECTOR, UNKNOWN, 
TIMESTAMP, DATE, BLOB, OBJECT`. Two levels must be kept apart (my earlier list 
ran them together) ⚠:
   
   **a. What can appear as a mapped value column** — `BOOLEAN, INT32, INT64, 
FLOAT, DOUBLE, TEXT, STRING, DATE, BLOB, OBJECT`. (`VECTOR` and `UNKNOWN` are 
enum members but are not produced on this path.)
   
   **b. `TIMESTAMP`** — in the table model there is no time column, so it never 
appears as a mapped value column. It arises only from the `-1` time-column 
placeholder in the tree model.
   
   Behaviour per type on the getters:
   
   - **`OBJECT`** — non-string getters throw `SQLException` with 
`OBJECT_ERR_MSG` (`IoTDBJDBCResultSet.java:74,313,335,385,406`); `getString` 
works via `BytesUtils.parseObjectByteArrayToString` 
(`IoTDBRpcDataSet.java:562-564`).
   - **`DATE`** — materialized as `INT32`; `getDate` is `getInt` + 
`DateUtils.parseIntToLocalDate` (`:597-601`), and `getString` formats via 
`DateUtils.formatDate` (`:568-569`).
   - **`TEXT` / `STRING`** — decoded with `TSFileConfig.STRING_CHARSET`, 
**not** with the connection charset (`:556-561`).
   - **`BLOB`** — returned as `getBinary(...).getValues()` (`:565-567`).
   - **When several logical columns share one physical column**, 
`dataTypeForTsBlockColumn` is written once per logical column (`:179`), so a 
later logical column **overwrites** the type recorded for that physical column. 
The current code does not detect or reject a conflict.
   - **Index constraint** — see §4: out-of-range indices do **not** currently 
raise `SQLException`, contrary to what my earlier version claimed.
   
   ## 3. Examples
   
   The server sends `columns = ["b","a"]`, `dataTypeList = ["INT32","INT32"]`, 
`columnIndex2TsBlockColumnIndexList = [1,0]`, and the physical TsBlock holds 
two INT32 columns with values `10` and `20`:
   
   ```java
   PreparedStatement ps = conn.prepareStatement("SELECT b, a FROM t WHERE a > 
?");
   ps.setInt(1, 0);
   ResultSet rs = ps.executeQuery();
   rs.next();
   rs.getInt(1);        // expected 20 (logical b → physical 1)
   rs.getInt("a");      // expected 10 (logical a → physical 0)
   ```
   
   | `columns` | mapping | physical values | expected `[col 1, col 2]` | 
expected by name |
   | --- | --- | --- | --- | --- |
   | `["b","a"]` | `[1,0]` | `[10, 20]` | `[20, 10]` | `b=20, a=10` |
   | `["x","y"]` | `[0,0]` | `[10]` | `[10, 10]` | `x=10, y=10` (must not 
throw) |
   | `["a","b"]` | `[0,1]` | `[10, 20]` | `[10, 20]` | identity case, 
regression control |
   
   Current behaviour for the same three rows: the identity mapping is 
substituted, so row 1 silently returns `[10, 20]` instead of `[20, 10]`, and 
row 2 resolves column 2 to `list.get(1)` = `1` and then calls 
`curTsBlock.getColumn(1)` on a one-column block, raising 
`ArrayIndexOutOfBoundsException`.
   
   **Server reachability.** This is not a synthetic-only fixture. 
`DatasetHeader.setTableColumnToTsBlockIndexMap` (`DatasetHeader.java:100-107`) 
builds the mapping by looking up each output symbol's position in the *child* 
output symbol list, so a projection order that differs from the scan order 
yields a non-identity mapping, and a symbol used twice yields duplicate entries.
   
   ## 4. Current divergence and fix
   
   `IoTDBTablePreparedStatement.processQueryResult()` 
(`iotdb-client/jdbc/src/main/java/org/apache/iotdb/jdbc/IoTDBTablePreparedStatement.java:215-232`)
 calls the 14-argument `IoTDBJDBCResultSet` constructor 
(`IoTDBJDBCResultSet.java:139-176`), which hard-codes `false` for `tableModel` 
and substitutes `IntStream.range(0, columnNameList.size())` for the mapping 
(`:174`). That constructor also never assigns `this.charset`, so the field 
keeps its initializer `TSFileConfig.STRING_CHARSET` (`:86`). Ordinary 
`IoTDBStatement` (`:541-558`) calls the full constructor and forwards all 
three, so the same response yields different results on the two paths.
   
   **Fix.** In `IoTDBTablePreparedStatement.java`, replace the result-set 
construction at `:218-232` with the full constructor, adding after `zoneId`:
   
   ```java
   charset,
   resp.isSetTableModel() && resp.isTableModel(),
   resp.getColumnIndex2TsBlockColumnIndexList()
   ```
   
   `charset` is already in scope — inherited from `IoTDBStatement` and assigned 
by this class's own constructor (`:92-94`) — and the call shape mirrors 
`IoTDBStatement.java:557-558`.
   
   ### Correction: the index / empty-slot requirement
   
   My earlier version demanded that "an out-of-range or empty slot raise 
`SQLException`, not `ArrayIndexOutOfBoundsException`", while also claiming 
`IoTDBRpcDataSet` needs no change. Those two statements cannot both hold, and 
the fix above does **not** deliver the first one. As it stands:
   
   - `getTsBlockColumnIndexForColumnIndex` (`IoTDBRpcDataSet.java:641-643`) is 
a bare `columnIndex2TsBlockColumnIndexList.get(columnIndex - 1)` — there is no 
bounds check on `columnIndex`.
   - The JDBC getters catch only `StatementExecutionException` (e.g. 
`IoTDBJDBCResultSet.java:537-543`), so an out-of-range ordinal surfaces as 
`IndexOutOfBoundsException`, an unknown column name as 
`IllegalArgumentException` (`:633-637`), and `findColumn` NPEs on an unknown 
name (`:617-619`).
   - The correct pattern already exists one method away: 
`findColumnNameByIndex` (`:621-629`) validates `<= 0` and `> 
columnNameList.size()` and throws `StatementExecutionException` with 
`COLUMN_INDEX_SHOULD_START_FROM_1` / `COLUMN_INDEX_OUT_OF_RANGE`.
   
   This should therefore be treated as a **separate decision**, and I would 
rather the maintainers make it explicitly than have it folded in silently:
   
   - **(a)** scope this fix to the constructor swap and drop the `SQLException` 
requirement, or
   - **(b)** additionally add the same bounds checks to 
`getTsBlockColumnIndexForColumnIndex` / `getTsBlockColumnIndexForColumnName` 
and translate them in the JDBC layer — a behaviour change for existing callers.
   
   ## 5. Acceptance
   
   1. All three rows in §3 return the expected values, identical to the 
`Statement` path for the same response.
   2. No `ArrayIndexOutOfBoundsException`.
   3. Every case asserts `getXxx(int)`, `getXxx(String)` **and the actual 
values** — asserting metadata alone does not pass, because the metadata itself 
is what is wrong.
   4. Both shapes are covered: reordered columns, and repeated 
projections/aliases.
   5. Test carriers: 
`iotdb-client/jdbc/src/test/java/org/apache/iotdb/jdbc/IoTDBTablePreparedStatementTest.java`
 for the mocked-RPC unit case, plus 
`integration-test/src/test/java/org/apache/iotdb/relational/it/db/it/IoTDBTablePreparedStatementJDBCIT.java`
 and `IoTDBTablePreparedStatementIT.java` against a real server, which is what 
actually demonstrates reachability.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to