PDGGK opened a new pull request, #1358:
URL: https://github.com/apache/paimon-rust/pull/1358

   ### Purpose
   
   Linked issue: close #1357
   
   A table partitioned by CHAR, DECIMAL or TIMESTAMP can be created, written 
and read, but `ALTER TABLE ... DROP PARTITION`, `TRUNCATE TABLE ... PARTITION` 
and `INSERT OVERWRITE ... PARTITION` all fail during planning when they name 
one of its partitions, for example with `Cannot convert 'ab' to Char(...)`. A 
DATE key only works with a quoted string: `DATE '2024-03-01'` fails with 
`Unsupported partition value expression`.
   
   The conversion from a SQL literal to a partition value only handled numbers, 
VARCHAR, quoted dates and booleans.
   
   ### Brief change log
   
   `sql_expr_to_datum` now converts quoted strings for CHAR, DECIMAL, TIMESTAMP 
and TIMESTAMP WITH LOCAL TIME ZONE keys, numbers for DECIMAL keys, and `DATE 
'...'` and `TIMESTAMP '...'` literals. It does this with 
`parse_partition_value` from `procedures.rs`, which the `CALL` procedures 
already use for the same types, so a value means the same thing in both places.
   
   `datum_to_constant_array` needed the same types for a static `INSERT 
OVERWRITE`. It now builds those columns through the existing `datum_to_scalar`.
   
   A value the key cannot hold fails with `Invalid partition value`. The 
existing rejection of `'7'` for an INT key and `'true'` for a BOOLEAN key is 
unchanged.
   
   ### Tests
   
   `test_partition_specs_on_char_decimal_timestamp_and_typed_literals` runs a 
static `INSERT OVERWRITE`, `TRUNCATE ... PARTITION` and `DROP PARTITION` 
against each key type and literal form through `SQLContext`, and checks the 
rows left in the table. 
`test_partition_spec_rejects_a_value_the_key_cannot_hold` covers a value too 
long for a CHAR key. On unmodified main the first fails with `Plan("Cannot 
convert 'ab' to Char(CharType { nullable: true, length: 3 })")`. Reverting one 
part of the change at a time fails it with the error for that part: `Cannot 
convert 12.50 to Decimal(...)` without the DECIMAL number arm, `Unsupported 
partition value expression: DATE '2024-03-01'` without the typed-literal arm, 
and `Unsupported datum type for partition column: DEC(1250,s2)` without the 
`datum_to_constant_array` change.
   
   The `sql_context::` and `procedures::` unit tests pass (205). So do 
`tests/sql_context_tests.rs` (73), `tests/pk_tables.rs` (55), 
`tests/table_type_routing.rs` (70) and `tests/time_travel_schema_tests.rs` (8) 
with `RUST_MIN_STACK=16777216` as CI sets it. `cargo +1.98.0 fmt --all -- 
--check` and `cargo +1.98.0 clippy --locked -p paimon-datafusion --all-targets 
--no-deps -- -D warnings` are clean.
   
   I ran these on Linux aarch64 in Docker, with a local filesystem catalog and 
the session time zone at its default. I did not run them against a REST 
catalog, and I did not check the result from a Java or Spark client. A 
TIMESTAMP WITH TIME ZONE key is only covered with `partition.legacy-name` at 
its default.
   
   ### API and Format
   
   No.
   
   ### Documentation
   
   No.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to