juntaozhang opened a new pull request, #9008:
URL: https://github.com/apache/paimon/pull/9008
### Purpose
this PR follow #8185, refactors partition time resolution as the title
suggests.
### Tests
#### Benchmark: `PartitionTimeResolver` vs `PartitionTimeExtractor`
- Env：
| Item | Value |
|--------|-----------------------------------------------|
| OS | Linux 7.0.0-28-generic (Ubuntu 26.04), x86_64 |
| CPU | Intel(R) Core(TM) Ultra 9 285H |
| JDK | OpenJDK 11.0.31 |
- `50,000` warmup iterations + `200,000` iterations for each task.
- Each result is reported as average time per operation.
- `Init` means constructing the object.
- `Ratio < 1` means `PartitionTimeResolver` is faster; `Ratio > 1` means
slower.
- Parse to date
| Scenario |
`PartitionTimeExtractor.extract` | `PartitionTimeResolver` | Ratio |
|------------------------------------------------------------|----------------------------------|----------------------------------------------|------------|
| `$dt` / `yyyyMM-dd` | 8.43
µs/op | 465 ns/op |
0.06x |
| `$y$m$d` / `yyyyMMdd` | 6.65
µs/op | 360 ns/op |
0.05x |
| `$dt $hr` / `yyyyMMdd HH` | 797 ns/op
| 524 ns/op | 0.66x
|
| `$a-$b-$c T $hour:$minute:00` / `yyyy-MM-dd 'T' HH:mm:ss` | 1.86
µs/op | 672 ns/op |
0.36x |
- Each loop construct the object and `parse` to date
| Scenario | Init +
`PartitionTimeExtractor.extract` | Init + `PartitionTimeResolver` | Ratio |
|------------------------------------------------------------|-----------------------------------------|-----------------------------------------------------|-----------|
| `$dt` / `yyyyMM-dd` | 6.27
µs/op | 621 ns/op
| 0.10x |
| `$y$m$d` / `yyyyMMdd` | 6.63
µs/op | 863 ns/op
| 0.13x |
| `$dt $hr` / `yyyyMMdd HH` | 786 ns/op
| 870 ns/op
| 1.11x |
| `$a-$b-$c T $hour:$minute:00` / `yyyy-MM-dd 'T' HH:mm:ss` | 1.85
µs/op | 8.59 µs/op
| 4.63x |
- Resolve date to partition values
| Scenario |
`ChainTableUtils.calPartValues` | `PartitionTimeResolver` | Ratio |
|------------------------------------------------------------|---------------------------------|------------------------------------------------|----------|
| `$dt` / `yyyyMM-dd` | 668 ns/op
| 244 ns/op | 0.37x
|
| `$y$m$d` / `yyyyMMdd` | 849 ns/op
| 300 ns/op | 0.35x
|
| `$dt $hr` / `yyyyMMdd HH` | 855 ns/op
| 261 ns/op | 0.31x
|
| `$a-$b-$c T $hour:$minute:00` / `yyyy-MM-dd 'T' HH:mm:ss` | 2.32
µs/op | 574 ns/op |
0.25x |
- Each loop construct the object and `resolve` date to partition values
| Scenario |
`ChainTableUtils.calPartValues` | Init + `PartitionTimeResolver` | Ratio |
|------------------------------------------------------------|---------------------------------|-------------------------------------------------------|-------|
| `$dt` / `yyyyMM-dd` | 668 ns/op
| 271 ns/op |
0.41x |
| `$y$m$d` / `yyyyMMdd` | 849 ns/op
| 655 ns/op |
0.77x |
| `$dt $hr` / `yyyyMMdd HH` | 855 ns/op
| 661 ns/op |
0.77x |
| `$a-$b-$c T $hour:$minute:00` / `yyyy-MM-dd 'T' HH:mm:ss` | 2.32
µs/op | 8.25 µs/op
| 3.56x |
#### Summary
- Hot path (Parse / Resolve): `PartitionTimeResolver` is significantly
faster, often 2–20x over `PartitionTimeExtractor`.
- Construction cost: `PartitionTimeResolver` is heavier because it validates
and maps pattern ↔ formatter upfront. `PartitionTimeExtractor` constructs
almost instantly because its do nothing.
- Overall: For common patterns, Init + Parse / Init + Resolve is still
faster with the `PartitionTimeResolver`. Only complex patterns become slower
overall due to recursive matching in construction.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
