SteNicholas opened a new pull request, #547:
URL: https://github.com/apache/paimon-cpp/pull/547

   ### Purpose
   
   Linked issue: close #540
   
   Complete the TIME type support in Paimon C++, following Paimon Java. TIME 
stays an Arrow `time32[ms]` holding the milliseconds of the day, with its 
declared precision in the `paimon.time.precision` field metadata.
   
   - Add `FieldType::TIME` and `Literal(FieldType::TIME, millis)`.
   - Read and write TIME as int32 in `BinaryRow`, serializers, comparators, the 
sort buffer, columnar rows and row-to-Arrow conversion, so primary-key tables 
support TIME as a primary key and as a value column.
   - Validate and evaluate TIME predicates (comparisons, `IN`/`NOT IN`, null 
checks), and push them down to Parquet and ORC.
   - Allow TIME partition keys. Partition values are formatted like Java 
`TimeToStringCastRule`, or as milliseconds with `partition.legacy-name`, and 
parsed like `BinaryStringUtils.toTime`. Partition value errors now name the 
partition key.
   - Hash TIME bucket keys as int32 in the default and Hive bucket functions 
and in bucket pruning.
   - Map TIME to an ORC `int` and to an Avro `int` with the `time-millis` 
logical type, rejecting Avro precisions above 3 like Java, and report TIME file 
statistics as int statistics (Parquet, ORC, Avro, Mosaic).
   - Reject `time32` units other than milliseconds in predicates, partition and 
bucket keys, `variant_get` targets, and ORC and Avro reads and writes.
   - Cast between TIME and STRING, TIMESTAMP or TIMESTAMP_LTZ in schema 
evolution, for both data and statistics, like the Java TIME cast rules. A TIME 
field is cast to STRING with the precision it declares, which also formats TIME 
values in system tables, and a TIMESTAMP_LTZ field is cast to TIME in the local 
time zone. `variant_get` casts strings and timestamps to TIME.
   - Support TIME in min/max aggregation, the realtime store and the hash index.
   
   Not covered yet, as documented: TIME column default values, file or global 
indexes on TIME columns, and verifying Paimon Java reading tables written by 
Paimon C++.
   
   ### Tests
   
   - Unit tests for TIME string parsing and formatting (precisions 0/3/6/9, 
offsets, out-of-range and malformed input), literals, literal conversion, 
predicate validation and evaluation, row writers, serializers, comparators, 
columnar rows, row-to-Arrow conversion, the sort buffer, aggregation and the 
row equalizer.
   - Partition computer tests for TIME partition keys of precisions 0/3/6/9, 
legacy partition names, default partitions and invalid values; bucket tests for 
the default and Hive bucket functions and bucket pruning.
   - Cast tests for STRING/TIMESTAMP/TIMESTAMP_LTZ to and from TIME for arrays 
and literals, the cast executor factory, `CastedRow`, field mapping, statistics 
evolution and `variant_get`, including non-millisecond targets.
   - Format tests for ORC, Avro, Parquet and Mosaic TIME reads, writes and 
statistics, and the realtime store and hash index.
   - Integration tests: primary-key tables with TIME partition, primary and 
bucket keys across buckets, with upserts, deletes, full compaction, bucket and 
statistics pruning, and deletion vectors; append-only tables with TIME 
partition and value predicates; TIME schema evolution; the `$files` system 
table; and reading Java-written TIME tables in ORC and Avro.
   
   ### API and Format
   
   - API: adds `FieldType::TIME` in `include/paimon/defs.h`, and 
`Literal(FieldType, int32_t)` now also accepts `FieldType::TIME`.
   - Format: TIME is written to ORC as `int` and to Avro as `int` with the 
`time-millis` logical type, both matching Paimon Java. TIME partition values 
and bucket ids follow Paimon Java.
   
   ### Documentation
   
   Yes. `docs/source/user_guide/data_types.rst` documents the TIME support of 
primary-key and append-only tables, partition and bucket keys, predicates, each 
file format and its statistics, string parsing, schema evolution casts, and 
what is not supported yet.
   
   ### Generative AI tooling
   
   Generated-by: Claude Code (Claude Opus 5.5)
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to