Hi everyone,

I'm happy to report that the Apache TsFile input format extension has been 
merged into Apache Druid: apache/druid#20729 landed on master on 2026-11-01. It 
will be available in the upcoming Druid release built from master.

Background

In August we discussed this with dev@druid and agreed to ship it as a Druid 
community extension under extensions-contrib, maintained and compatibility 
tested on the TsFile side:

https://lists.apache.org/thread/qmrkhobp7p07wkmcf4y4myq36k866zz0

That is what was merged. The module is extensions-contrib/tsfile-extensions 
(load it as druid-tsfile-extensions), and it adds an input format of type 
tsfile so Druid batch ingestion can read TsFile files directly — no export 
step, no intermediate Parquet/CSV.

What it supports
Reads table-model TsFiles through the TsFile Java library 2.5.0 
(TsFileReaderBuilder -> ITsFileReader -> ResultSet).
Both ingestion paths: classic batch (index_parallel) and SQL-based ingestion 
via MSQ EXTERN.
Options: tables and columns to select a subset, and interval for a time range. 
interval is pushed down into the reader, which uses the per-chunk time 
statistics to skip chunks outside the range.
TsFile's implicit timestamp is exposed as the column Time (epoch millis).

Minimal inputFormat:

{
  "type": "tsfile",
  "tables": ["sensor"],
  "columns": ["region", "device", "temperature", "humidity", "status"]
}

and in the data schema:

"timestampSpec": { "column": "Time", "format": "millis" }

The full user documentation is in the Druid repo at 
docs/development/extensions-contrib/tsfile.md, including a working sample file 
and ingestion spec under extensions-contrib/tsfile-extensions/examples/.

Known limitations in this first version
Table model only. Files written with the legacy tree model (device paths such 
as root.sg.d1.s1 with no table schema in the footer) are not supported yet.
The input format is not splittable; parallelism comes from the input source, 
one file per task.
Druid does not pass a task's time window to input formats, so the interval 
option has to be set explicitly.
VECTOR, OBJECT, ROW and UNKNOWN columns fail the task with a clear error. BLOB 
is returned as a Base64 string and DATE as epoch days.
One ingestion spec reads one row schema, so a file with several tables of 
different shapes should be ingested table by table.
What's next

We are the maintainers of this integration, so please treat it as ours: bug 
reports, compatibility issues and feature requests are welcome on this list or 
as issues/PRs against apache/druid, and we will pick them up.

It would be great to get feedback from anyone who already has TsFile data — 
especially large files, tree-model files, and non-local input sources (S3/GCS), 
which are the areas we would most like to harden next.

Thanks to the Druid community, and to Frank Chen for the careful review that 
got this through.

Best regards, 
Colinlee

Reply via email to