MxlIzumi opened a new issue, #68392:
URL: https://github.com/apache/doris/issues/68392
### Summary
When reading a DATETIME column from SelectDB 2.1.8-3 using the Doris Flink
Connector over Thrift, version 26.3.0 produces an incorrect 1970 timestamp.
Switching the test task to Connector 25.0.0 restores the expected value.
This is a report against a SelectDB distribution, not a claim that the same
behavior has already been reproduced on stock Apache Doris 2.1.8.
### Environment
- Flink: company distribution based on 1.20
- Server: SelectDB 2.1.8-3
- Artifact: org.apache.doris:flink-doris-connector-1.20
- Affected version tested: 26.3.0
- Working version tested: 25.0.0
- Read protocol: Thrift; FE arrow_flight_sql_port is -1, and Flight SQL was
not used
- Date/time comparison below uses UTC+08:00
- Doris DATETIME mapped to Flink TIMESTAMP(0)
### Observed behavior
| Value | ctime / mtime |
| --- | --- |
| Direct query in Doris | 2026-07-30 19:44:28 |
| Flink Connector 26.3.0 | 1970-01-01 08:29:45.411868 |
| Flink Connector 25.0.0 after switching versions | 2026-07-30 19:44:28 |
The incorrect value is present in the raw TaskManager output, not just the
platform's result viewer. Casting the Flink timestamp to STRING still gives the
incorrect value. Other fields are read successfully.
### SQL shape used for testing (anonymized)
The source is an existing Doris OLAP table with BIGINT, VARCHAR and DATETIME
columns. Table names, identifiers and connection details below have been
replaced with placeholders; this exact standalone example has not been
independently rerun on stock Apache Doris.
~~~sql
CREATE TABLE doris_datetime_probe (
id BIGINT,
name STRING,
ctime TIMESTAMP(0),
mtime TIMESTAMP(0)
) WITH (
'connector' = 'doris',
'fenodes' = '<FE_HOST>:<HTTP_PORT>',
'table.identifier' = 'test.datetime_probe',
'username' = '<USER>',
'password' = '<PASSWORD>'
);
CREATE TABLE datetime_probe_print (
id BIGINT,
name STRING,
ctime STRING,
mtime STRING
) WITH (
'connector' = 'print'
);
INSERT INTO datetime_probe_print
SELECT id, name, CAST(ctime AS STRING), CAST(mtime AS STRING)
FROM doris_datetime_probe
WHERE id = 1;
~~~
Use a source row whose ctime and mtime are 2026-07-30 19:44:28, and compare
the output between the two Connector versions. The id/name in the example are
anonymized.
### Source-code observations / suspected cause
The official published source packages show a relevant change in
org.apache.doris.flink.serialization.RowBatch:
- In 25.0.0, getDateTime() delegates to longToLocalDateTime(long), which
infers seconds / milliseconds / microseconds from the numeric magnitude. Its
comment states: "Currently, the scale of doris's arrow datetimev2 is hardcoded
to 6, and there is also a time zone problem in arrow, so use timestamp to
convert first" (approximately lines 554–576).
- In 26.3.0, getDateTime() uses the Arrow timestamp metadata. For timestamps
with timezone metadata it calls longToLocalDateTime(value,
timestampType.getUnit(), DEFAULT_ZONE_ID); the MICROSECOND branch divides by
1,000,000 (approximately lines 702–741). For timestamps without timezone
metadata it delegates to vector.getObject().
In UTC+08:00, the expected date corresponds to epoch seconds 1785411868.
Interpreting that same integer as microseconds produces exactly 1970-01-01
08:29:45.411868.
This suggests a mismatch between the actual numeric unit and the declared
Arrow unit, which the older magnitude-based conversion may have tolerated. We
have NOT yet captured the raw Arrow value/unit/timezone or the loaded class
location, so this is a hypothesis rather than a confirmed wire-level diagnosis.
We also have not identified the first affected Connector release between 25.0.0
and 26.3.0.
Sources:
- [25.0.0 official
sources](https://repo.maven.apache.org/maven2/org/apache/doris/flink-doris-connector-1.20/25.0.0/flink-doris-connector-1.20-25.0.0-sources.jar)
- [26.3.0 official base-module
sources](https://repo.maven.apache.org/maven2/org/apache/doris/flink-doris-connector-base/26.3.0/flink-doris-connector-base-26.3.0-sources.jar)
### Questions
1. Is this a known server/Connector compatibility issue or regression in
handling legacy Arrow timestamp encoding?
2. Which Doris/SelectDB versions and read protocols are affected?
3. Is there a fix or recommended Connector/server combination? Is there a
safe compatibility option rather than relying on timestamp magnitude heuristics?
4. Could the compatibility documentation mention any applicable
restrictions? The broad Doris version range does not make this behavior
apparent.
Happy to provide additional sanitized diagnostics. Please advise if this
should be tracked in the Connector repository or with the SelectDB distribution
instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
