PDGGK opened a new pull request, #1358:
URL: https://github.com/apache/paimon-rust/pull/1358
### Purpose
Linked issue: close #1357
A table partitioned by CHAR, DECIMAL or TIMESTAMP can be created, written
and read, but `ALTER TABLE ... DROP PARTITION`, `TRUNCATE TABLE ... PARTITION`
and `INSERT OVERWRITE ... PARTITION` all fail during planning when they name
one of its partitions, for example with `Cannot convert 'ab' to Char(...)`. A
DATE key only works with a quoted string: `DATE '2024-03-01'` fails with
`Unsupported partition value expression`.
The conversion from a SQL literal to a partition value only handled numbers,
VARCHAR, quoted dates and booleans.
### Brief change log
`sql_expr_to_datum` now converts quoted strings for CHAR, DECIMAL, TIMESTAMP
and TIMESTAMP WITH LOCAL TIME ZONE keys, numbers for DECIMAL keys, and `DATE
'...'` and `TIMESTAMP '...'` literals. It does this with
`parse_partition_value` from `procedures.rs`, which the `CALL` procedures
already use for the same types, so a value means the same thing in both places.
`datum_to_constant_array` needed the same types for a static `INSERT
OVERWRITE`. It now builds those columns through the existing `datum_to_scalar`.
A value the key cannot hold fails with `Invalid partition value`. The
existing rejection of `'7'` for an INT key and `'true'` for a BOOLEAN key is
unchanged.
### Tests
`test_partition_specs_on_char_decimal_timestamp_and_typed_literals` runs a
static `INSERT OVERWRITE`, `TRUNCATE ... PARTITION` and `DROP PARTITION`
against each key type and literal form through `SQLContext`, and checks the
rows left in the table.
`test_partition_spec_rejects_a_value_the_key_cannot_hold` covers a value too
long for a CHAR key. On unmodified main the first fails with `Plan("Cannot
convert 'ab' to Char(CharType { nullable: true, length: 3 })")`. Reverting one
part of the change at a time fails it with the error for that part: `Cannot
convert 12.50 to Decimal(...)` without the DECIMAL number arm, `Unsupported
partition value expression: DATE '2024-03-01'` without the typed-literal arm,
and `Unsupported datum type for partition column: DEC(1250,s2)` without the
`datum_to_constant_array` change.
The `sql_context::` and `procedures::` unit tests pass (205). So do
`tests/sql_context_tests.rs` (73), `tests/pk_tables.rs` (55),
`tests/table_type_routing.rs` (70) and `tests/time_travel_schema_tests.rs` (8)
with `RUST_MIN_STACK=16777216` as CI sets it. `cargo +1.98.0 fmt --all --
--check` and `cargo +1.98.0 clippy --locked -p paimon-datafusion --all-targets
--no-deps -- -D warnings` are clean.
I ran these on Linux aarch64 in Docker, with a local filesystem catalog and
the session time zone at its default. I did not run them against a REST
catalog, and I did not check the result from a Java or Spark client. A
TIMESTAMP WITH TIME ZONE key is only covered with `partition.legacy-name` at
its default.
### API and Format
No.
### Documentation
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]