sundapeng opened a new pull request, #697:
URL: https://github.com/apache/paimon-rust/pull/697
### Problem
The three count columns and `total_buckets` of the `$partitions` system
table were declared
non-nullable, so a statistic the catalog never had reported to it came out
as `-1` and read like a
measurement: the partition holds -1 rows, -1 files, -1 buckets. Reporting
`0` instead would be worse,
because `0` is a real measurement meaning the partition is empty.
This is the same failure mode as the file-level row count fixed in #624 — a
placeholder presented as
an exact statistic — arriving through a second entrance, the catalog. A
catalog-managed format table
registers partitions without necessarily measuring them, so an unreported
statistic is the normal
case there, not an edge case.
### Fix
`Partition` gains `UNKNOWN`, `UNKNOWN_TOTAL_BUCKETS` and `is_known()`,
documenting that on the
observation plane any negative value means never measured while zero is an
exact zero — and saying
nothing about the delta plane, where a negative value is a decrement to
apply. That mirrors what the
crate already does for file-level counts with
`DataFileMeta::ROW_COUNT_UNKNOWN` and
`row_count_known()`.
The four columns become nullable and unknown renders as NULL.
### Tests
-
`spec::partition::tests::test_unknown_is_negative_and_zero_is_a_measurement`
pins the contract:
any negative is unknown, zero and positives are measurements.
-
`sql_context_tests::test_partitions_system_table_shows_unreported_statistics_as_null`
asserts real
statistics **still arrive as values** before flipping the stub catalog to
unknown, so a NULL in the
second half means unknown rather than the column having stopped being
populated at all. It also
checks the partition is still listed — only its statistics are unknown.
Reverting the `record_count` field to non-nullable makes that test fail with
`Column 'record_count' is declared as non-nullable but contains null
values`, so the assertion is
load-bearing.
`spec::partition` 32 passed, `sql_context_tests` 55 passed, `system_tables`
15 passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
