This is an automated email from the ASF dual-hosted git repository.
HTHou pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/iotdb-client-nodejs.git
The following commit(s) were added to refs/heads/develop by this push:
new 31601ee fix: DATE values encoded as days-since-epoch instead of INT32
yyyyMMdd (#15)
31601ee is described below
commit 31601eee4a6d63e11abd9c7bc30f2f37ae93860d
Author: CritasWang <[email protected]>
AuthorDate: Mon Jul 13 14:27:34 2026 +0800
fix: DATE values encoded as days-since-epoch instead of INT32 yyyyMMdd (#15)
* fix: encode DATE as INT32 yyyyMMdd instead of days-since-epoch
IoTDB's DATE wire format is an INT32 encoded as year*10000 + month*100 + day
(e.g. 2026-07-13 -> 20260713), matching the Java client's
DateUtils.parseDateExpressionToInt and the C# client. The Node.js client was
writing and reading days-since-epoch, so DATE values written via tablets
were
corrupted and unreadable by other clients (and vice versa).
Adjudicated against a live IoTDB 2.0.6 server: a row inserted via SQL
literal
'2026-07-13' reads back as INT32 20260713 on the wire.
Changes:
- Add shared parseDateToInt / parseIntToDate utils in src/utils/DataTypes.ts
(UTC calendar components, matching Java LocalDate semantics) and export
them
- Session.serializeColumn case 9: write yyyyMMdd (was days-since-epoch)
- FastSerializer.serializeDateColumn: same fix on the fast path
- Session.deserializeColumn case 9 (TSQueryDataSet): decode yyyyMMdd -> Date
- ColumnDecoder Int32ArrayColumnDecoder: decode DATE (type 9) columns to
Date
- Session.parseTsBlock: DATE columns arrive typed as INT32 (1) in the
TsBlock
header; convert them to Date using the metadata dataTypeList
- Unit tests: yyyyMMdd round-trip, exact wire bytes for 2026-07-13
(0x01352769), TsBlock decode; updated existing DATE serialization tests
Verified: tablet-written Date and SQL-literal date now read back identical
against a live IoTDB 2.0.6 (both 2026-07-13T00:00:00.000Z).
* fix: address review feedback on DATE yyyyMMdd encoding
Addresses three review comments from HTHou on PR #15:
1. parseTsBlock DATE conversion now respects
columnIndex2TsBlockColumnIndexList. dataTypeList is ordered by LOGICAL
response columns while physical TsBlock columns may be deduplicated or
reordered, so converting physical column i based on dataTypes[i] could
convert the wrong column. The mapping (logical -> physical, -1 = time
column) is now passed into parseQueryResult from both call sites
(executeQueryStatement initial batch and SessionDataSet.fetchNextBatch)
and inverted into per-physical-column logical types. DATE conversion
only applies when a physical column's logical type is unambiguously
DATE; conflicting duplicate mappings log a warning and skip conversion
instead of crashing. Absent mapping keeps identity behavior.
2. Docs updated to the yyyyMMdd contract: docs/data-types.md and the
English/Chinese tree user guides no longer describe numeric DATE as
days since epoch; they now document the INT32 yyyyMMdd encoding with
examples (20240101 for 2024-01-01).
3. parseDateToInt/parseIntToDate now validate like the Java client's
DateUtils/LocalDate: invalid Date objects (NaN time), non-integer or
non-finite numbers, years outside 1000-9999, and impossible calendar
dates (e.g. 20230229, month 13, day 0) throw a descriptive Error
instead of being silently normalized or serialized as garbage.
null/undefined handling is unchanged (callers filter nulls first).
Unit tests added: non-identity mapping TsBlock conversion (crafted
TsBlock buffers with INT32 wire columns + DATE logical type), dedup and
conflicting-mapping cases, and validation boundary tests (leap day
20240229 ok, 20230229/20241301/20240100/year 999/year 10000/NaN
Date/non-integer/legacy days-epoch 19723 all throw).
---
docs/data-types.md | 6 +-
docs/user-guide-tree-zh.md | 2 +-
docs/user-guide-tree.md | 2 +-
src/client/ColumnDecoder.ts | 13 +-
src/client/Session.ts | 112 +++++++++++--
src/client/SessionDataSet.ts | 1 +
src/index.ts | 7 +-
src/utils/DataTypes.ts | 119 +++++++++++++-
src/utils/FastSerializer.ts | 18 +-
tests/unit/DataTypes.test.ts | 292 +++++++++++++++++++++++++++++++++
tests/unit/FastSerializer.test.ts | 27 ++-
tests/unit/TabletSerialization.test.ts | 31 +++-
12 files changed, 578 insertions(+), 52 deletions(-)
diff --git a/docs/data-types.md b/docs/data-types.md
index 55d08f3..4589a0e 100644
--- a/docs/data-types.md
+++ b/docs/data-types.md
@@ -17,7 +17,7 @@ The IoTDB Node.js client supports all standard IoTDB data
types as defined in Ap
| 6 | VECTOR | Vector data (not yet implemented) | - | - |
| 7 | UNKNOWN | Unknown type (reserved) | - | - |
| 8 | TIMESTAMP | Timestamp (milliseconds) | `Date` | 8 bytes |
-| 9 | DATE | Date (days since epoch) | `Date` | 4 bytes |
+| 9 | DATE | Calendar date (INT32 `yyyyMMdd`, e.g. `20240101` for 2024-01-01)
| `Date` | 4 bytes |
| 10 | BLOB | Binary data | `Buffer` | Variable (4-byte length + content) |
| 11 | STRING | UTF-8 encoded string | `string` | Variable (4-byte length +
content) |
| 12 | OBJECT | Object type (reserved) | - | - |
@@ -176,7 +176,7 @@ for (const row of result.rows) {
| `bigint` | INT64, TIMESTAMP | Direct mapping |
| `string` | TEXT, STRING | UTF-8 encoded |
| `Buffer` | BLOB | Binary data |
-| `Date` | DATE, TIMESTAMP | Converted to days or milliseconds |
+| `Date` | DATE, TIMESTAMP | DATE: encoded as `yyyyMMdd` integer (e.g.
`20240101`); TIMESTAMP: milliseconds since epoch |
### IoTDB to JavaScript
@@ -190,7 +190,7 @@ for (const row of result.rows) {
| TEXT | `string` | UTF-8 decoded |
| BLOB | `Buffer` | Raw binary data |
| STRING | `string` | UTF-8 decoded |
-| DATE | `Date` | Days since epoch converted to Date |
+| DATE | `Date` | `yyyyMMdd` integer (e.g. `20240101`) converted to Date at
UTC midnight |
| TIMESTAMP | `Date` | Milliseconds since epoch |
## Null Values
diff --git a/docs/user-guide-tree-zh.md b/docs/user-guide-tree-zh.md
index 3b67d93..695603d 100644
--- a/docs/user-guide-tree-zh.md
+++ b/docs/user-guide-tree-zh.md
@@ -350,7 +350,7 @@ const pool = new SessionPool(poolConfig);
| 4 | DOUBLE | number | 64 位浮点数 |
| 5 | TEXT | string | UTF-8 字符串 |
| 8 | TIMESTAMP | number/Date | 自纪元以来的毫秒数 |
-| 9 | DATE | number/Date | 自纪元以来的天数 |
+| 9 | DATE | number/Date | 日历日期,`yyyyMMdd` 整数(例如 `20240101` 表示 2024-01-01)|
| 10 | BLOB | Buffer | 二进制数据 |
| 11 | STRING | string | 与 TEXT 相同 |
diff --git a/docs/user-guide-tree.md b/docs/user-guide-tree.md
index 10173a0..83ac5b2 100644
--- a/docs/user-guide-tree.md
+++ b/docs/user-guide-tree.md
@@ -350,7 +350,7 @@ The tree model supports all IoTDB data types:
| 4 | DOUBLE | number | 64-bit floating point |
| 5 | TEXT | string | UTF-8 string |
| 8 | TIMESTAMP | number/Date | Milliseconds since epoch |
-| 9 | DATE | number/Date | Days since epoch |
+| 9 | DATE | number/Date | Calendar date as `yyyyMMdd` integer (e.g.
`20240101` for 2024-01-01) |
| 10 | BLOB | Buffer | Binary data |
| 11 | STRING | string | Same as TEXT |
diff --git a/src/client/ColumnDecoder.ts b/src/client/ColumnDecoder.ts
index 52701cc..25d5c13 100644
--- a/src/client/ColumnDecoder.ts
+++ b/src/client/ColumnDecoder.ts
@@ -18,6 +18,7 @@
*/
import { logger } from "../utils/Logger";
+import { parseIntToDate } from "../utils/DataTypes";
/**
* Column encoding types matching Apache IoTDB ColumnEncoding enum
@@ -152,7 +153,6 @@ class Int32ArrayColumnDecoder implements ColumnDecoder {
switch (dataType) {
case 1: // INT32
- case 9: // DATE
for (let i = 0; i < positionCount; i++) {
if (nullIndicators && nullIndicators[i]) {
values[i] = null;
@@ -163,6 +163,17 @@ class Int32ArrayColumnDecoder implements ColumnDecoder {
}
break;
+ case 9: // DATE (INT32 yyyyMMdd encoding -> Date object)
+ for (let i = 0; i < positionCount; i++) {
+ if (nullIndicators && nullIndicators[i]) {
+ values[i] = null;
+ continue;
+ }
+ values[i] = parseIntToDate(buffer.readInt32BE(currentOffset));
+ currentOffset += 4;
+ }
+ break;
+
case 3: // FLOAT
for (let i = 0; i < positionCount; i++) {
if (nullIndicators && nullIndicators[i]) {
diff --git a/src/client/Session.ts b/src/client/Session.ts
index 879f62e..5c8cffc 100644
--- a/src/client/Session.ts
+++ b/src/client/Session.ts
@@ -35,6 +35,7 @@ import {
serializeTimestamps
} from "../utils/FastSerializer";
import { globalBufferPool } from "../utils/BufferPool";
+import { parseDateToInt, parseIntToDate } from "../utils/DataTypes";
const ttypes = require("../thrift/generated/client_types");
@@ -318,6 +319,7 @@ export class Session {
response.columns?.length || 0,
response.dataTypeList || [],
ignoreTimeStamp,
+ response.columnIndex2TsBlockColumnIndexList,
);
} else if (response.queryDataSet) {
// Old columnar format (TSQueryDataSet)
@@ -741,18 +743,11 @@ export class Session {
return buffer;
}
case 9: {
- // DATE (stored as INT32 - days since epoch) - Use big-endian
+ // DATE (stored as INT32 - yyyyMMdd, e.g. 20240101) - Use big-endian
const buffer = Buffer.alloc(values.length * 4);
values.forEach((v, i) => {
- let days = 0;
- if (v !== null && v !== undefined) {
- if (v instanceof Date) {
- days = Math.floor(v.getTime() / (24 * 60 * 60 * 1000));
- } else {
- days = v;
- }
- }
- buffer.writeInt32BE(days, i * 4);
+ const encoded = v === null || v === undefined ? 0 :
parseDateToInt(v);
+ buffer.writeInt32BE(encoded, i * 4);
});
return buffer;
}
@@ -888,12 +883,16 @@ export class Session {
* - Value columns data
*
* @param ignoreTimeStamp - If true, no time column is present
+ * @param columnIndex2TsBlockColumnIndexList - Server-provided mapping from
+ * LOGICAL response column index to PHYSICAL TsBlock column index (-1 =
+ * time column); identity mapping is assumed when absent
*/
async parseQueryResult(
queryResult: Buffer[],
_columnCount: number,
dataTypes: string[],
ignoreTimeStamp: boolean = false,
+ columnIndex2TsBlockColumnIndexList?: number[],
): Promise<any[][]> {
const rows: any[][] = [];
@@ -907,6 +906,14 @@ export class Session {
);
logger.debug(`parseQueryResult: dataTypes: ${JSON.stringify(dataTypes)}`);
+ // dataTypes is ordered by LOGICAL response columns; physical TsBlock
+ // columns may be deduplicated/reordered. Derive each PHYSICAL column's
+ // logical type once so parseTsBlock can convert DATE columns correctly.
+ const physicalColumnTypes = this.derivePhysicalColumnTypes(
+ dataTypes,
+ columnIndex2TsBlockColumnIndexList,
+ );
+
// Process each TsBlock in queryResult
for (let blockIndex = 0; blockIndex < queryResult.length; blockIndex++) {
const tsBlockBuffer = Buffer.isBuffer(queryResult[blockIndex])
@@ -920,7 +927,7 @@ export class Session {
try {
const blockRows = this.parseTsBlock(
tsBlockBuffer,
- dataTypes,
+ physicalColumnTypes,
ignoreTimeStamp,
);
rows.push(...blockRows);
@@ -934,6 +941,55 @@ export class Session {
return rows;
}
+ /**
+ * Derive the per-PHYSICAL-TsBlock-column logical data types from the
+ * logically-ordered dataTypeList and the server-provided
+ * columnIndex2TsBlockColumnIndexList (logical index -> physical index,
+ * -1 = time column).
+ *
+ * Multiple logical columns may map to the same physical column (the server
+ * deduplicates identical output columns). In practice duplicates share one
+ * type; if they ever disagree, the physical column's type is left undefined
+ * so no type-specific conversion is applied.
+ *
+ * When the mapping is absent, the logical order is the physical order
+ * (identity), preserving the previous behavior.
+ */
+ private derivePhysicalColumnTypes(
+ dataTypes: string[],
+ columnIndex2TsBlockColumnIndexList?: number[],
+ ): (string | undefined)[] {
+ if (
+ !columnIndex2TsBlockColumnIndexList ||
+ columnIndex2TsBlockColumnIndexList.length === 0
+ ) {
+ return dataTypes;
+ }
+
+ const physicalTypes: (string | undefined)[] = [];
+ const conflicting = new Set<number>();
+ for (let i = 0; i < dataTypes.length; i++) {
+ const physicalIndex = columnIndex2TsBlockColumnIndexList[i];
+ // -1 marks the time column; ignore missing/negative entries
+ if (physicalIndex === undefined || physicalIndex < 0) {
+ continue;
+ }
+ const existing = physicalTypes[physicalIndex];
+ if (existing === undefined) {
+ physicalTypes[physicalIndex] = dataTypes[i];
+ } else if (existing !== dataTypes[i]) {
+ conflicting.add(physicalIndex);
+ }
+ }
+ for (const physicalIndex of conflicting) {
+ logger.warn(
+ `derivePhysicalColumnTypes: conflicting logical types for TsBlock
column ${physicalIndex}; skipping type conversion for it`,
+ );
+ physicalTypes[physicalIndex] = undefined;
+ }
+ return physicalTypes;
+ }
+
/**
* Parse a single TsBlock buffer
* TsBlock format (from Apache IoTDB C# client):
@@ -947,10 +1003,14 @@ export class Session {
* regardless of the ignoreTimeStamp setting. The ignoreTimeStamp flag only
* affects whether the timestamp is included in the returned row data, not
the
* TsBlock binary format. This matches the behavior of iotdb-client-csharp.
+ *
+ * @param physicalColumnTypes - Logical data types ordered by PHYSICAL
+ * TsBlock column (see derivePhysicalColumnTypes); an undefined entry means
+ * "unknown, apply no type-specific conversion"
*/
private parseTsBlock(
buffer: Buffer,
- dataTypes: string[],
+ physicalColumnTypes: (string | undefined)[],
ignoreTimeStamp: boolean,
): any[][] {
let offset = 0;
@@ -1021,6 +1081,25 @@ export class Session {
);
}
+ // DATE columns arrive in TsBlock with wire type INT32 (1); the real DATE
+ // type is only present in the query metadata. Convert those columns'
+ // yyyyMMdd integers (e.g. 20240101) to Date objects here. Only convert
+ // when the physical column's logical type is unambiguously DATE.
+ for (let i = 0; i < valueColumns.length; i++) {
+ if (
+ valueColumnTypes[i] === 1 &&
+ physicalColumnTypes[i] !== undefined &&
+ this.getDataTypeCode(physicalColumnTypes[i]) === 9
+ ) {
+ const colValues = valueColumns[i].values;
+ for (let j = 0; j < colValues.length; j++) {
+ if (colValues[j] !== null) {
+ colValues[j] = parseIntToDate(colValues[j]);
+ }
+ }
+ }
+ }
+
// Build rows from columns
const rows: any[][] = [];
for (let rowIndex = 0; rowIndex < positionCount; rowIndex++) {
@@ -1313,15 +1392,14 @@ export class Session {
break;
}
case 9: {
- // DATE (stored as INT32 - days since epoch) - TSQueryDataSet uses
BIG ENDIAN
+ // DATE (stored as INT32 - yyyyMMdd) - TSQueryDataSet uses BIG ENDIAN
for (let i = 0; i < rowCount; i++) {
if (this.isNull(bitmap, i)) {
values.push(null);
} else {
- // Convert days since epoch to Date object
- const days = buffer.readInt32BE(i * 4);
- const date = new Date(days * 24 * 60 * 60 * 1000);
- values.push(date);
+ // Convert yyyyMMdd integer to Date object
+ const encoded = buffer.readInt32BE(i * 4);
+ values.push(parseIntToDate(encoded));
}
}
break;
diff --git a/src/client/SessionDataSet.ts b/src/client/SessionDataSet.ts
index 346ddde..ecaeb8a 100644
--- a/src/client/SessionDataSet.ts
+++ b/src/client/SessionDataSet.ts
@@ -274,6 +274,7 @@ export class SessionDataSet {
this.columnNames.length,
this.columnTypes,
this.ignoreTimeStamp,
+ this.columnIndex2TsBlockColumnIndexList,
);
} else if (response.queryDataSet) {
// Old columnar format (TSQueryDataSet)
diff --git a/src/index.ts b/src/index.ts
index 54fdee4..76a2965 100644
--- a/src/index.ts
+++ b/src/index.ts
@@ -39,7 +39,12 @@ export {
parseNodeUrls,
} from "./utils/Config";
export { logger, LogLevel } from "./utils/Logger";
-export { TSDataType, getDataTypeName } from "./utils/DataTypes";
+export {
+ TSDataType,
+ getDataTypeName,
+ parseDateToInt,
+ parseIntToDate,
+} from "./utils/DataTypes";
export { RedirectException, TSStatusCode } from "./utils/Errors";
export { RedirectCache } from "./client/RedirectCache";
export { enableGlobalCleanup } from "./utils/ProcessCleanup";
diff --git a/src/utils/DataTypes.ts b/src/utils/DataTypes.ts
index a497e8b..6fd2de3 100644
--- a/src/utils/DataTypes.ts
+++ b/src/utils/DataTypes.ts
@@ -77,8 +77,8 @@ export enum TSDataType {
/**
* Date with day precision (no time component)
- * JavaScript type: Date or number (days since epoch)
- * Storage size: 4 bytes (stored as INT32)
+ * JavaScript type: Date or number (yyyyMMdd integer, e.g. 20240101)
+ * Storage size: 4 bytes (stored as INT32, encoded as year*10000 + month*100
+ day)
*/
DATE = 9,
@@ -99,6 +99,121 @@ export enum TSDataType {
// OBJECT = 12, // Reserved - not yet implemented
}
+/**
+ * Days per month (index 0 = January) in a non-leap year.
+ */
+const DAYS_IN_MONTH = [31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31];
+
+function isLeapYear(year: number): boolean {
+ return (year % 4 === 0 && year % 100 !== 0) || year % 400 === 0;
+}
+
+/**
+ * Validate a calendar date, matching the Java client's DateUtils semantics:
+ * years are restricted to 1000-9999 and (year, month, day) must form a real
+ * calendar date (LocalDate would reject e.g. 2023-02-29).
+ *
+ * @throws Error if the components do not form a valid calendar date
+ */
+function validateCalendarDate(
+ year: number,
+ month: number,
+ day: number,
+ source: string,
+): void {
+ if (year < 1000 || year > 9999) {
+ throw new Error(
+ `Invalid DATE value ${source}: year ${year} is out of range [1000,
9999]`,
+ );
+ }
+ if (month < 1 || month > 12) {
+ throw new Error(
+ `Invalid DATE value ${source}: month ${month} is out of range [1, 12]`,
+ );
+ }
+ const maxDay =
+ month === 2 && isLeapYear(year) ? 29 : DAYS_IN_MONTH[month - 1];
+ if (day < 1 || day > maxDay) {
+ throw new Error(
+ `Invalid DATE value ${source}: day ${day} is out of range [1, ${maxDay}]
for ${year}-${String(month).padStart(2, "0")}`,
+ );
+ }
+}
+
+/**
+ * Convert a JavaScript Date (or an already-encoded yyyyMMdd number) to the
+ * IoTDB DATE wire format: an INT32 encoded as year*10000 + month*100 + day
+ * (e.g. 2024-01-01 -> 20240101).
+ *
+ * This matches the Java client's DateUtils.parseDateExpressionToInt and the
+ * C# client. The calendar date is taken from the Date's UTC components,
+ * consistent with `new Date("2024-01-01")` which parses as UTC midnight.
+ *
+ * Invalid inputs are rejected (like the Java client, which limits years to
+ * 1000-9999 and whose LocalDate guarantees a valid calendar date): invalid
+ * Date objects, non-finite/non-integer numbers, and numbers whose yyyyMMdd
+ * decomposition is not a real calendar date all throw.
+ *
+ * Note: null/undefined are not handled here — callers filter nulls before
+ * invoking this helper.
+ *
+ * @param value - Date object or a yyyyMMdd integer (validated, then passed
+ * through unchanged)
+ * @returns The yyyyMMdd integer encoding
+ * @throws Error if the value is not a valid calendar date
+ */
+export function parseDateToInt(value: Date | number): number {
+ if (value instanceof Date) {
+ if (isNaN(value.getTime())) {
+ throw new Error("Invalid DATE value: Date object is invalid (NaN time)");
+ }
+ const year = value.getUTCFullYear();
+ const month = value.getUTCMonth() + 1;
+ const day = value.getUTCDate();
+ validateCalendarDate(year, month, day, value.toISOString());
+ return year * 10000 + month * 100 + day;
+ }
+ if (!Number.isInteger(value)) {
+ throw new Error(
+ `Invalid DATE value ${value}: expected an integer in yyyyMMdd form (e.g.
20240101)`,
+ );
+ }
+ validateCalendarDate(
+ Math.trunc(value / 10000),
+ Math.trunc(value / 100) % 100,
+ value % 100,
+ String(value),
+ );
+ return value;
+}
+
+/**
+ * Convert an IoTDB DATE wire value (INT32, year*10000 + month*100 + day)
+ * back to a JavaScript Date at UTC midnight of that calendar date.
+ *
+ * Inverse of {@link parseDateToInt}; matches the Java client's
+ * DateUtils.parseIntToDate. The value is validated the same way as
+ * {@link parseDateToInt} — non-integer numbers, years outside 1000-9999,
+ * and impossible calendar dates (e.g. 20230229) throw instead of being
+ * silently normalized by the Date constructor.
+ *
+ * @param value - The yyyyMMdd integer (e.g. 20240101)
+ * @returns Date at UTC midnight of the encoded calendar date
+ * @throws Error if the value is not a valid yyyyMMdd calendar date
+ */
+export function parseIntToDate(value: number): Date {
+ if (!Number.isInteger(value)) {
+ throw new Error(
+ `Invalid DATE value ${value}: expected an integer in yyyyMMdd form (e.g.
20240101)`,
+ );
+ }
+ const year = Math.trunc(value / 10000);
+ const month = Math.trunc(value / 100) % 100;
+ const day = value % 100;
+ validateCalendarDate(year, month, day, String(value));
+ return new Date(Date.UTC(year, month - 1, day));
+}
+
/**
* Get the name of a TSDataType from its numeric code
* @param typeCode - The numeric type code
diff --git a/src/utils/FastSerializer.ts b/src/utils/FastSerializer.ts
index a87f9b3..76ae9ed 100644
--- a/src/utils/FastSerializer.ts
+++ b/src/utils/FastSerializer.ts
@@ -18,6 +18,7 @@
*/
import { globalBufferPool } from "./BufferPool";
+import { parseDateToInt } from "./DataTypes";
/**
* Fast serialization utilities for IoTDB data types
@@ -167,26 +168,19 @@ export function serializeTimestampColumn(values: any[]):
Buffer {
}
/**
- * Serialize DATE column (4 bytes per value, days since epoch)
+ * Serialize DATE column (4 bytes per value, INT32 yyyyMMdd encoding)
* Optimized: Single buffer allocation with direct writes
*/
export function serializeDateColumn(values: any[]): Buffer {
const size = values.length * 4;
const buffer = size >= 1024 ? globalBufferPool.acquire(size) :
Buffer.allocUnsafe(size);
-
+
for (let i = 0; i < values.length; i++) {
const v = values[i];
- let days = 0;
- if (v !== null && v !== undefined) {
- if (v instanceof Date) {
- days = Math.floor(v.getTime() / (24 * 60 * 60 * 1000));
- } else {
- days = v;
- }
- }
- buffer.writeInt32BE(days, i * 4);
+ const encoded = v === null || v === undefined ? 0 : parseDateToInt(v);
+ buffer.writeInt32BE(encoded, i * 4);
}
-
+
return buffer.subarray(0, size);
}
diff --git a/tests/unit/DataTypes.test.ts b/tests/unit/DataTypes.test.ts
new file mode 100644
index 0000000..fd300a2
--- /dev/null
+++ b/tests/unit/DataTypes.test.ts
@@ -0,0 +1,292 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import {
+ parseDateToInt,
+ parseIntToDate,
+} from "../../src/utils/DataTypes";
+import {
+ BaseColumnDecoder,
+ ColumnEncoding,
+} from "../../src/client/ColumnDecoder";
+import { Session } from "../../src/client/Session";
+
+describe("DATE yyyyMMdd conversion", () => {
+ describe("parseDateToInt", () => {
+ it("should encode a Date as INT32 yyyyMMdd (year*10000 + month*100 +
day)", () => {
+ expect(parseDateToInt(new Date("2026-07-13"))).toBe(20260713);
+ expect(parseDateToInt(new Date("2024-01-01"))).toBe(20240101);
+ expect(parseDateToInt(new Date("2024-12-31"))).toBe(20241231);
+ // Leap day
+ expect(parseDateToInt(new Date("2024-02-29"))).toBe(20240229);
+ });
+
+ it("should pass through already-encoded valid numbers unchanged", () => {
+ expect(parseDateToInt(20260713)).toBe(20260713);
+ expect(parseDateToInt(10000101)).toBe(10000101); // year lower bound
+ expect(parseDateToInt(99991231)).toBe(99991231); // year upper bound
+ });
+
+ it("should accept the leap day of a leap year", () => {
+ expect(parseDateToInt(20240229)).toBe(20240229);
+ expect(parseDateToInt(new Date("2024-02-29"))).toBe(20240229);
+ });
+
+ it("should reject invalid Date objects", () => {
+ expect(() => parseDateToInt(new Date(NaN))).toThrow(/Invalid DATE/);
+ });
+
+ it("should reject Date objects with years outside 1000-9999", () => {
+ expect(() => parseDateToInt(new Date("0999-12-31"))).toThrow(
+ /year 999/,
+ );
+ });
+
+ it("should reject non-finite and non-integer numbers", () => {
+ expect(() => parseDateToInt(NaN)).toThrow(/Invalid DATE/);
+ expect(() => parseDateToInt(Infinity)).toThrow(/Invalid DATE/);
+ expect(() => parseDateToInt(20240101.5)).toThrow(/Invalid DATE/);
+ });
+
+ it("should reject numbers that are not real calendar dates", () => {
+ expect(() => parseDateToInt(20230229)).toThrow(/day 29/); // not a leap
year
+ expect(() => parseDateToInt(20241301)).toThrow(/month 13/);
+ expect(() => parseDateToInt(20240100)).toThrow(/day 0/);
+ expect(() => parseDateToInt(9991231)).toThrow(/year 999/);
+ expect(() => parseDateToInt(100000101)).toThrow(/year 10000/);
+ // Old days-since-epoch value: 19723 -> year 1, month 97 -> invalid
+ expect(() => parseDateToInt(19723)).toThrow(/Invalid DATE/);
+ expect(() => parseDateToInt(0)).toThrow(/Invalid DATE/);
+ });
+ });
+
+ describe("parseIntToDate", () => {
+ it("should decode INT32 yyyyMMdd to a Date at UTC midnight", () => {
+ const date = parseIntToDate(20260713);
+ expect(date.getUTCFullYear()).toBe(2026);
+ expect(date.getUTCMonth()).toBe(6); // July (0-based)
+ expect(date.getUTCDate()).toBe(13);
+ expect(date.getUTCHours()).toBe(0);
+ expect(date.getUTCMinutes()).toBe(0);
+ });
+
+ it("should decode the leap day of a leap year", () => {
+ const date = parseIntToDate(20240229);
+ expect(date.getUTCFullYear()).toBe(2024);
+ expect(date.getUTCMonth()).toBe(1);
+ expect(date.getUTCDate()).toBe(29);
+ });
+
+ it("should reject impossible calendar dates instead of normalizing", () =>
{
+ // new Date(Date.UTC(2023, 1, 29)) would silently become 2023-03-01
+ expect(() => parseIntToDate(20230229)).toThrow(/day 29/);
+ expect(() => parseIntToDate(20241301)).toThrow(/month 13/);
+ expect(() => parseIntToDate(20240100)).toThrow(/day 0/);
+ });
+
+ it("should reject years outside 1000-9999", () => {
+ expect(() => parseIntToDate(9991231)).toThrow(/year 999/);
+ expect(() => parseIntToDate(100000101)).toThrow(/year 10000/);
+ // Old days-since-epoch value is not a valid yyyyMMdd
+ expect(() => parseIntToDate(19723)).toThrow(/Invalid DATE/);
+ });
+
+ it("should reject non-integer numbers", () => {
+ expect(() => parseIntToDate(NaN)).toThrow(/Invalid DATE/);
+ expect(() => parseIntToDate(20240101.5)).toThrow(/Invalid DATE/);
+ });
+ });
+
+ describe("round-trip", () => {
+ it("should round-trip Date -> yyyyMMdd -> Date", () => {
+ const dates = [
+ new Date("1970-01-01"),
+ new Date("2000-02-29"),
+ new Date("2024-01-01"),
+ new Date("2026-07-13"),
+ new Date("9999-12-31"),
+ ];
+ for (const original of dates) {
+ const encoded = parseDateToInt(original);
+ const decoded = parseIntToDate(encoded);
+ expect(decoded.getTime()).toBe(original.getTime());
+ // And the integer round-trips too
+ expect(parseDateToInt(decoded)).toBe(encoded);
+ }
+ });
+ });
+
+ describe("TsBlock column decode (Int32ArrayColumnDecoder)", () => {
+ it("should decode a DATE column value as a Date from yyyyMMdd wire bytes",
() => {
+ // Column layout: 1 byte null flag (0 = no nulls) + INT32 BE values
+ // 20260713 = 0x01352769
+ const buffer = Buffer.from([0x00, 0x01, 0x35, 0x27, 0x69]);
+ const decoder = BaseColumnDecoder.getDecoder(ColumnEncoding.Int32Array);
+ const { column, bytesRead } = decoder.readColumn(buffer, 0, 9, 1);
+
+ expect(bytesRead).toBe(5);
+ const value = column.values[0];
+ expect(value).toBeInstanceOf(Date);
+ expect(value.getUTCFullYear()).toBe(2026);
+ expect(value.getUTCMonth()).toBe(6);
+ expect(value.getUTCDate()).toBe(13);
+ });
+
+ it("should still decode plain INT32 columns as numbers", () => {
+ const buffer = Buffer.from([0x00, 0x01, 0x35, 0x27, 0x69]);
+ const decoder = BaseColumnDecoder.getDecoder(ColumnEncoding.Int32Array);
+ const { column } = decoder.readColumn(buffer, 0, 1, 1);
+ expect(column.values[0]).toBe(20260713);
+ });
+ });
+
+ describe("parseTsBlock DATE conversion with
columnIndex2TsBlockColumnIndexList", () => {
+ let session: Session;
+
+ beforeEach(() => {
+ session = new Session({
+ host: "localhost",
+ port: 6667,
+ username: "root",
+ password: "root",
+ });
+ });
+
+ /**
+ * Craft a TsBlock buffer with INT32 (wire type 1) value columns:
+ * valueColumnCount i32 BE | type bytes | positionCount i32 BE |
+ * time encoding byte | value encoding bytes | time column | value columns
+ * Each column: 1-byte null flag (0 = no nulls) + big-endian values.
+ */
+ function craftInt32TsBlock(
+ timestamps: number[],
+ int32Columns: number[][],
+ ): Buffer {
+ const positionCount = timestamps.length;
+ const parts: Buffer[] = [];
+
+ const header = Buffer.alloc(4);
+ header.writeInt32BE(int32Columns.length, 0);
+ parts.push(header);
+ // Wire type of every value column is INT32 (1)
+ parts.push(Buffer.from(int32Columns.map(() => 1)));
+
+ const posBuf = Buffer.alloc(4);
+ posBuf.writeInt32BE(positionCount, 0);
+ parts.push(posBuf);
+
+ // Encodings: time column Int64Array (2), value columns Int32Array (1)
+ parts.push(Buffer.from([2, ...int32Columns.map(() => 1)]));
+
+ // Time column: null flag + INT64 BE timestamps
+ const timeBuf = Buffer.alloc(1 + positionCount * 8);
+ timeBuf.writeUInt8(0, 0);
+ timestamps.forEach((t, i) => {
+ timeBuf.writeBigInt64BE(BigInt(t), 1 + i * 8);
+ });
+ parts.push(timeBuf);
+
+ // Value columns: null flag + INT32 BE values
+ for (const columnValues of int32Columns) {
+ const colBuf = Buffer.alloc(1 + positionCount * 4);
+ colBuf.writeUInt8(0, 0);
+ columnValues.forEach((v, i) => {
+ colBuf.writeInt32BE(v, 1 + i * 4);
+ });
+ parts.push(colBuf);
+ }
+
+ return Buffer.concat(parts);
+ }
+
+ it("should convert the DATE column per the non-identity logical->physical
mapping", async () => {
+ // Logical columns: [DATE, INT32]; physical TsBlock columns reordered:
+ // logical 0 (DATE) -> physical 1, logical 1 (INT32) -> physical 0
+ const buffer = craftInt32TsBlock([1000], [[42], [20240101]]);
+
+ const rows = await (session as any).parseQueryResult(
+ [buffer],
+ 2,
+ ["DATE", "INT32"],
+ false,
+ [1, 0],
+ );
+
+ expect(rows).toHaveLength(1);
+ const [timestamp, physical0, physical1] = rows[0];
+ expect(Number(timestamp)).toBe(1000);
+ // Physical column 0 is logically INT32 -> stays a number
+ expect(physical0).toBe(42);
+ // Physical column 1 is logically DATE -> converted to Date
+ expect(physical1).toBeInstanceOf(Date);
+ expect(physical1.getUTCFullYear()).toBe(2024);
+ expect(physical1.getUTCMonth()).toBe(0);
+ expect(physical1.getUTCDate()).toBe(1);
+ });
+
+ it("should keep identity behavior when the mapping is absent", async () =>
{
+ const buffer = craftInt32TsBlock([1000], [[20240101], [42]]);
+
+ const rows = await (session as any).parseQueryResult(
+ [buffer],
+ 2,
+ ["DATE", "INT32"],
+ false,
+ );
+
+ expect(rows).toHaveLength(1);
+ expect(rows[0][1]).toBeInstanceOf(Date);
+ expect(rows[0][1].getUTCFullYear()).toBe(2024);
+ expect(rows[0][2]).toBe(42);
+ });
+
+ it("should handle deduplicated columns (two logical columns -> one
physical)", async () => {
+ // SELECT d, d: logical [DATE, DATE] both map to physical 0
+ const buffer = craftInt32TsBlock([1000], [[20240101]]);
+
+ const rows = await (session as any).parseQueryResult(
+ [buffer],
+ 2,
+ ["DATE", "DATE"],
+ false,
+ [0, 0],
+ );
+
+ expect(rows).toHaveLength(1);
+ expect(rows[0][1]).toBeInstanceOf(Date);
+ expect(rows[0][1].getUTCFullYear()).toBe(2024);
+ });
+
+ it("should not crash nor convert when logical types disagree for one
physical column", async () => {
+ const buffer = craftInt32TsBlock([1000], [[20240101]]);
+
+ const rows = await (session as any).parseQueryResult(
+ [buffer],
+ 2,
+ ["DATE", "INT32"],
+ false,
+ [0, 0],
+ );
+
+ expect(rows).toHaveLength(1);
+ // Ambiguous type -> no DATE conversion applied
+ expect(rows[0][1]).toBe(20240101);
+ });
+ });
+});
diff --git a/tests/unit/FastSerializer.test.ts
b/tests/unit/FastSerializer.test.ts
index 4f2b7ba..f6e673d 100644
--- a/tests/unit/FastSerializer.test.ts
+++ b/tests/unit/FastSerializer.test.ts
@@ -155,18 +155,31 @@ describe("FastSerializer", () => {
});
describe("DATE Serialization", () => {
- it("should serialize DATE values correctly", () => {
+ it("should serialize DATE values as INT32 yyyyMMdd", () => {
const date1 = new Date("2024-01-01");
- const days1 = Math.floor(date1.getTime() / (24 * 60 * 60 * 1000));
- const values = [date1, 100, null, undefined];
+ const values = [date1, 20241231, null, undefined];
const buffer = serializeDateColumn(values);
expect(buffer.length).toBe(16); // 4 * 4 bytes
- expect(buffer.readInt32BE(0)).toBe(days1);
- expect(buffer.readInt32BE(4)).toBe(100);
+ expect(buffer.readInt32BE(0)).toBe(20240101); // yyyyMMdd, NOT days
since epoch
+ expect(buffer.readInt32BE(4)).toBe(20241231); // numbers pass through
unchanged
expect(buffer.readInt32BE(8)).toBe(0); // null -> 0
expect(buffer.readInt32BE(12)).toBe(0); // undefined -> 0
});
+
+ it("should produce exact wire bytes for 2026-07-13", () => {
+ // IoTDB DATE wire format: INT32 yyyyMMdd, big-endian
+ // 20260713 = 0x01352769
+ const buffer = serializeDateColumn([new Date("2026-07-13")]);
+ expect(Array.from(buffer)).toEqual([0x01, 0x35, 0x27, 0x69]);
+ });
+
+ it("should reject invalid DATE values", () => {
+ expect(() => serializeDateColumn([20230229])).toThrow(/Invalid DATE/);
// not a leap year
+ expect(() => serializeDateColumn([new Date(NaN)])).toThrow(
+ /Invalid DATE/,
+ );
+ });
});
describe("BLOB Serialization", () => {
@@ -241,8 +254,8 @@ describe("FastSerializer", () => {
const tsBuffer = serializeColumnFast([1000, 2000], 8);
expect(tsBuffer.length).toBe(16);
- // DATE (9)
- const dateBuffer = serializeColumnFast([100, 200], 9);
+ // DATE (9) - values must be valid yyyyMMdd integers
+ const dateBuffer = serializeColumnFast([20240101, 20241231], 9);
expect(dateBuffer.length).toBe(8);
// BLOB (10)
diff --git a/tests/unit/TabletSerialization.test.ts
b/tests/unit/TabletSerialization.test.ts
index fb86706..620a432 100644
--- a/tests/unit/TabletSerialization.test.ts
+++ b/tests/unit/TabletSerialization.test.ts
@@ -342,18 +342,35 @@ describe('Tablet Serialization', () => {
expect(buffer.readDoubleBE(16)).toBe(0.0);
});
- test('should serialize DATE column', () => {
+ test('should serialize DATE column as INT32 yyyyMMdd', () => {
const date1 = new Date('2024-01-01');
const date2 = new Date('2024-12-31');
- const values = [date1, date2, 0];
+ const values = [date1, date2, 20260713];
const buffer = (session as any).serializeColumn(values, TSDataType.DATE);
expect(buffer.length).toBe(12); // 3 values * 4 bytes
- const days1 = Math.floor(date1.getTime() / (24 * 60 * 60 * 1000));
- const days2 = Math.floor(date2.getTime() / (24 * 60 * 60 * 1000));
- expect(buffer.readInt32BE(0)).toBe(days1);
- expect(buffer.readInt32BE(4)).toBe(days2);
- expect(buffer.readInt32BE(8)).toBe(0);
+ expect(buffer.readInt32BE(0)).toBe(20240101); // yyyyMMdd, NOT days
since epoch
+ expect(buffer.readInt32BE(4)).toBe(20241231);
+ expect(buffer.readInt32BE(8)).toBe(20260713); // numbers pass through
unchanged
+ });
+
+ test('should reject invalid DATE values during serialization', () => {
+ expect(() =>
+ (session as any).serializeColumn([20230229], TSDataType.DATE),
+ ).toThrow(/Invalid DATE/); // 2023 is not a leap year
+ expect(() =>
+ (session as any).serializeColumn([new Date(NaN)], TSDataType.DATE),
+ ).toThrow(/Invalid DATE/);
+ });
+
+ test('should serialize DATE 2026-07-13 with exact wire bytes', () => {
+ // IoTDB DATE wire format: INT32 yyyyMMdd, big-endian
+ // 20260713 = 0x01352769
+ const buffer = (session as any).serializeColumn(
+ [new Date('2026-07-13')],
+ TSDataType.DATE,
+ );
+ expect(Array.from(buffer)).toEqual([0x01, 0x35, 0x27, 0x69]);
});
test('should serialize TIMESTAMP column', () => {
