DaZuiZui commented on issue #18846:
URL: https://github.com/apache/iotdb/issues/18846#issuecomment-6160193918
# Corrected functional definition: result column mapping for Table-Model
`PreparedStatement`
> It adds no grammar and changes no parsing rule. What follows is the
**result-column-mapping semantics** of already-existing syntax on the JDBC
client, i.e. the behaviour contract the fix satisfies.
## 1. Syntax definition (affected surface)
The mapping semantics apply to every Table Model query using `?`
placeholders, in particular the three shapes below:
```sql
-- (a) projection order differs from the materialized order
SELECT b, a FROM t WHERE a > ?;
-- (b) repeated projection / aliases sharing one physical column
SELECT a AS x, a AS y FROM t WHERE a > ?;
-- (c) parameter in WHERE, time column in the projection
SELECT time, s1 FROM metrics WHERE device = ? AND s1 > ?;
```
Key points:
- The projection list fixes the **logical result-column order**, which may
differ from the **physical column order** of the server's TsBlock.
- The correspondence between the two is determined solely by the mapping the
server sends; the client adopts it and never derives it on its own.
- One physical column may be referenced by several logical columns (case
(b)); the server may therefore send repeated entries.
## 2. Functional description
### 2.1 Components and their meanings
| Component | Source | Meaning |
| --- | --- | --- |
| `columns` | `resp.getColumns()` | Logical result column names; their order
is the result-set column order |
| `columnIndex2TsBlockColumnIndexList` | `resp` | **Authoritative mapping**:
logical column ordinal → physical TsBlock column index |
| `columnNameIndexMap` | `resp.columnNameIndexMap` | Not consumed by the
client. `IoTDBRpcDataSet` accepts it as the `columnNameIndex` parameter
(`IoTDBRpcDataSet.java:99`) and never reads it again. The name → ordinal lookup
actually used is `columnOrdinalMap` (`:61`, *"used because the server returns
deduplicated columns"*), built internally from `columnNameList` (`:181-184`)
with first-occurrence-wins. Passing this field alone therefore cannot preserve
the mapping — which is exactly the point of §4 |
| `dataTypeList` | `resp.getDataTypeList()` | Type declaration **per logical
column, not per physical column**. It is indexed by logical ordinal —
`IoTDBRpcDataSet.java:173` reads `columnTypeList.get(i)` where `i` walks
`columnNameList` — and it is explicitly **not deduplicated** (`:59`). Its
length equals `columns.size()`, which is *not* the physical block width |
| `tableModel` | `resp.isSetTableModel() && resp.isTableModel()` |
Table-model flag. Note it is currently **accepted and discarded**:
`IoTDBRpcDataSet.java:112` declares the parameter and never stores or reads it.
Forwarding it is still required for parity with `IoTDBStatement` |
| `ignoreTimeStamp` | `resp` | Whether the tree-model time column is hidden.
Per `IoTDBRpcDataSet.java:114`, *"only used for tree model, table model this
field will always be true"* — so no `-1` entry is prepended |
| `charset` | the connection | Charset used by the `getBytes(...)` getters
(`IoTDBJDBCResultSet.java:391,412`) |
### 2.2 Rules
1. **Adopt the mapping as sent.** The client uses the server-provided
`columnIndex2TsBlockColumnIndexList` verbatim and never rebuilds it as an
identity mapping.
2. **Duplicates are allowed.** `m[i]` values may repeat; a repeated logical
column returns the value of the physical column it refers to.
3. **Physical block width** is `max(m) + 1`
(`IoTDBRpcDataSet.java:163-166`); the client sizes its column-type arrays from
that, **not** from the number of logical columns.
4. **Index-based and name-based reads share the same mapping** and return
identical values.
5. **Duplicate logical column names resolve to the first occurrence**
(`:181-184`).
6. Only when the mapping is absent does the client fall back to the existing
derivation (tree model: `-1` placeholder for the time column plus a shift;
identity otherwise).
7. `tableModel` comes from `resp.isSetTableModel() && resp.isTableModel()`
and is never hard-coded.
### 2.3 Types
`dataTypeList` carries one of the `TSDataType` names. The full enum is
`BOOLEAN, INT32, INT64, FLOAT, DOUBLE, TEXT, STRING, VECTOR, UNKNOWN,
TIMESTAMP, DATE, BLOB, OBJECT`. Two levels stay apart:
**a. What can appear as a mapped value column** — `BOOLEAN, INT32, INT64,
FLOAT, DOUBLE, TEXT, STRING, DATE, BLOB, OBJECT`. (`VECTOR` and `UNKNOWN` are
enum members that are not produced on this path.)
**b. `TIMESTAMP`** — in the table model there is no time column, so it never
appears as a mapped value column. It arises only from the `-1` time-column
placeholder in the tree model.
Behaviour per type on the getters:
- **`OBJECT`** — non-string getters throw `SQLException` with
`OBJECT_ERR_MSG` (`IoTDBJDBCResultSet.java:74,313,335,385,406`); `getString`
works via `BytesUtils.parseObjectByteArrayToString`
(`IoTDBRpcDataSet.java:562-564`).
- **`DATE`** — materialized as `INT32`; `getDate` is `getInt` +
`DateUtils.parseIntToLocalDate` (`:597-601`), and `getString` formats via
`DateUtils.formatDate` (`:568-569`).
- **`TEXT` / `STRING`** — decoded with `TSFileConfig.STRING_CHARSET`,
**not** with the connection charset (`:556-561`).
- **`BLOB`** — returned as `getBinary(...).getValues()` (`:565-567`).
- **When several logical columns share one physical column**,
`dataTypeForTsBlockColumn` is written once per logical column (`:179`), so a
later logical column overwrites the type recorded for that physical column. The
current code does not detect or reject such a conflict.
- **Index constraint** — see §4: out-of-range indices do **not** currently
raise `SQLException`.
## 3. Examples
The server sends `columns = ["b","a"]`, `dataTypeList = ["INT32","INT32"]`,
`columnIndex2TsBlockColumnIndexList = [1,0]`, and the physical TsBlock holds
two INT32 columns with values `10` and `20`:
```java
PreparedStatement ps = conn.prepareStatement("SELECT b, a FROM t WHERE a >
?");
ps.setInt(1, 0);
ResultSet rs = ps.executeQuery();
rs.next();
rs.getInt(1); // expected 20 (logical b → physical 1)
rs.getInt("a"); // expected 10 (logical a → physical 0)
```
| `columns` | mapping | physical values | expected `[col 1, col 2]` |
expected by name |
| --- | --- | --- | --- | --- |
| `["b","a"]` | `[1,0]` | `[10, 20]` | `[20, 10]` | `b=20, a=10` |
| `["x","y"]` | `[0,0]` | `[10]` | `[10, 10]` | `x=10, y=10` (must not
throw) |
| `["a","b"]` | `[0,1]` | `[10, 20]` | `[10, 20]` | identity case,
regression control |
Current behaviour for the same three rows: the identity mapping is
substituted, so row 1 silently returns `[10, 20]` instead of `[20, 10]`, and
row 2 resolves column 2 to `list.get(1)` = `1` and then calls
`curTsBlock.getColumn(1)` on a one-column block, raising
`ArrayIndexOutOfBoundsException`.
**Server reachability.** This is not a synthetic-only fixture.
`DatasetHeader.setTableColumnToTsBlockIndexMap` (`DatasetHeader.java:100-107`)
builds the mapping by looking up each output symbol's position in the *child*
output symbol list, so a projection order that differs from the scan order
yields a non-identity mapping, and a symbol used twice yields duplicate entries.
## 4. Current divergence and fix
`IoTDBTablePreparedStatement.processQueryResult()`
(`iotdb-client/jdbc/src/main/java/org/apache/iotdb/jdbc/IoTDBTablePreparedStatement.java:215-232`)
calls the 14-argument `IoTDBJDBCResultSet` constructor
(`IoTDBJDBCResultSet.java:139-181`), which hard-codes `false` for `tableModel`
(`:173`) and substitutes `IntStream.range(0, columnNameList.size())` for the
mapping (`:174`). That constructor also never assigns `this.charset`, so the
field keeps its initializer `TSFileConfig.STRING_CHARSET` (`:86`). Ordinary
`IoTDBStatement` (`:541-558`) calls the full constructor and forwards all
three, so the same response yields different results on the two paths.
**Fix.** In `IoTDBTablePreparedStatement.java`, replace the result-set
construction at `:218-232` with the full constructor, adding after `zoneId`:
```java
charset,
resp.isSetTableModel() && resp.isTableModel(),
new ArrayList<>(resp.getColumnIndex2TsBlockColumnIndexList())
```
`charset` is already in scope — inherited from `IoTDBStatement` and assigned
by this class's own constructor (`:92-94`) — and the call shape mirrors
`IoTDBStatement.java:557-558`. The mapping is passed as a defensive copy
because `IoTDBRpcDataSet` prepends `-1` in place when `!ignoreTimeStamp`
(`:167-170`) and the thrift getter hands back the response's internal list
(`TSExecuteStatementResp.java:815-817`).
### On the index / empty-slot requirement
This fix resolves the issue in full, and it does so without any separate
decision on bounds checking. My earlier version demanded that "an out-of-range
or empty slot raise `SQLException`, not `ArrayIndexOutOfBoundsException`", and
that requirement is hereby dropped rather than deferred. The reason it is
unnecessary is the invariant of the mapping itself:
- `getTsBlockColumnIndexForColumnIndex` (`IoTDBRpcDataSet.java:641-643`) is
a bare `columnIndex2TsBlockColumnIndexList.get(columnIndex - 1)`.
- The physical block width is `max(m) + 1`, so every valid logical ordinal
`i` satisfies `0 <= m[i] <= width - 1`. Once the mapping is adopted as sent,
`curTsBlock.getColumn(m[i])` cannot be out of range.
- The `ArrayIndexOutOfBoundsException` reported in §3 is an artifact of the
identity mapping being applied to a one-column block. It disappears together
with the mapping itself.
Bounds validation for out-of-range ordinals, unknown column names, and
`findColumn` (`IoTDBJDBCResultSet.java:537-543,617-619,633-637`) is a
robustness improvement that stands on its own and does not gate this fix.
`findColumnNameByIndex` (`:621-629`) already shows the pattern.
The 14-argument `IoTDBJDBCResultSet` constructor
(`IoTDBJDBCResultSet.java:139-181`) has no remaining callers after this change.
## 5. Acceptance
1. All three rows in §3 return the expected values, identical to the
`Statement` path for the same response.
2. No `ArrayIndexOutOfBoundsException`.
3. Every case asserts `getXxx(int)`, `getXxx(String)` **and the actual
values** — asserting metadata alone does not pass, because the metadata itself
is what is wrong.
4. Both shapes are covered: reordered columns, and repeated
projections/aliases.
5. Test carriers: the mocked-RPC unit cases extend
`iotdb-client/jdbc/src/test/java/org/apache/iotdb/jdbc/IoTDBTablePreparedStatementTest.java`,
and the real-server cases extend
`integration-test/src/test/java/org/apache/iotdb/relational/it/db/it/IoTDBTablePreparedStatementJDBCIT.java`
and `IoTDBTablePreparedStatementIT.java`, which is what demonstrates
reachability.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]