SteNicholas opened a new pull request, #547: URL: https://github.com/apache/paimon-cpp/pull/547
### Purpose Linked issue: close #540 Complete the TIME type support in Paimon C++, following Paimon Java. TIME stays an Arrow `time32[ms]` holding the milliseconds of the day, with its declared precision in the `paimon.time.precision` field metadata. - Add `FieldType::TIME` and `Literal(FieldType::TIME, millis)`. - Read and write TIME as int32 in `BinaryRow`, serializers, comparators, the sort buffer, columnar rows and row-to-Arrow conversion, so primary-key tables support TIME as a primary key and as a value column. - Validate and evaluate TIME predicates (comparisons, `IN`/`NOT IN`, null checks), and push them down to Parquet and ORC. - Allow TIME partition keys. Partition values are formatted like Java `TimeToStringCastRule`, or as milliseconds with `partition.legacy-name`, and parsed like `BinaryStringUtils.toTime`. Partition value errors now name the partition key. - Hash TIME bucket keys as int32 in the default and Hive bucket functions and in bucket pruning. - Map TIME to an ORC `int` and to an Avro `int` with the `time-millis` logical type, rejecting Avro precisions above 3 like Java, and report TIME file statistics as int statistics (Parquet, ORC, Avro, Mosaic). - Reject `time32` units other than milliseconds in predicates, partition and bucket keys, `variant_get` targets, and ORC and Avro reads and writes. - Cast between TIME and STRING, TIMESTAMP or TIMESTAMP_LTZ in schema evolution, for both data and statistics, like the Java TIME cast rules. A TIME field is cast to STRING with the precision it declares, which also formats TIME values in system tables, and a TIMESTAMP_LTZ field is cast to TIME in the local time zone. `variant_get` casts strings and timestamps to TIME. - Support TIME in min/max aggregation, the realtime store and the hash index. Not covered yet, as documented: TIME column default values, file or global indexes on TIME columns, and verifying Paimon Java reading tables written by Paimon C++. ### Tests - Unit tests for TIME string parsing and formatting (precisions 0/3/6/9, offsets, out-of-range and malformed input), literals, literal conversion, predicate validation and evaluation, row writers, serializers, comparators, columnar rows, row-to-Arrow conversion, the sort buffer, aggregation and the row equalizer. - Partition computer tests for TIME partition keys of precisions 0/3/6/9, legacy partition names, default partitions and invalid values; bucket tests for the default and Hive bucket functions and bucket pruning. - Cast tests for STRING/TIMESTAMP/TIMESTAMP_LTZ to and from TIME for arrays and literals, the cast executor factory, `CastedRow`, field mapping, statistics evolution and `variant_get`, including non-millisecond targets. - Format tests for ORC, Avro, Parquet and Mosaic TIME reads, writes and statistics, and the realtime store and hash index. - Integration tests: primary-key tables with TIME partition, primary and bucket keys across buckets, with upserts, deletes, full compaction, bucket and statistics pruning, and deletion vectors; append-only tables with TIME partition and value predicates; TIME schema evolution; the `$files` system table; and reading Java-written TIME tables in ORC and Avro. ### API and Format - API: adds `FieldType::TIME` in `include/paimon/defs.h`, and `Literal(FieldType, int32_t)` now also accepts `FieldType::TIME`. - Format: TIME is written to ORC as `int` and to Avro as `int` with the `time-millis` logical type, both matching Paimon Java. TIME partition values and bucket ids follow Paimon Java. ### Documentation Yes. `docs/source/user_guide/data_types.rst` documents the TIME support of primary-key and append-only tables, partition and bucket keys, predicates, each file format and its statistics, string parsing, schema evolution casts, and what is not supported yet. ### Generative AI tooling Generated-by: Claude Code (Claude Opus 5.5) 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
