binary-signal opened a new pull request, #3529:
URL: https://github.com/apache/fluss/pull/3529
### Purpose
Linked issue: close #3527
This PR removes the per-record cost which improves write throughput for
fluss log tables.
### Brief change log
Two patches:
1. Skip the dynamic-partition check when the table has no partition keys (
`fluss-client/.../WriterClient.java`):
```java
if (!tableInfo.getPartitionKeys().isEmpty()) {
dynamicPartitionCreator.checkAndCreatePartitionAsync(
physicalTablePath,
tableInfo.getPartitionKeys(),
tableInfo.getTableConfig().getAutoPartitionStrategy());
}
```
`DynamicPartitionCreator.checkAndCreatePartitionAsync` already returns
immediately when `partitionName == null`. This just hoists the same check
up
to the caller so the expensive argument never has to be evaluated. No
behaviour change.
2. Memoise the strategy on `TableConfig` (
`fluss-common/.../config/TableConfig.java`):
```java
public class TableConfig {
private final Configuration config;
private volatile AutoPartitionStrategy autoPartitionStrategy;
public AutoPartitionStrategy getAutoPartitionStrategy() {
AutoPartitionStrategy s = autoPartitionStrategy;
if (s == null) {
s = AutoPartitionStrategy.from(config);
autoPartitionStrategy = s; // benign race: same value
}
return s;
}
...
}
```
`TableConfig` is immutable after construction. `AutoPartitionStrategy` is
too with all fields `final`. A volatile field is enough.
### Tests
No new unit tests. The change is observable only as a perf improvement.
### Benchmark Performance Improvements
#### Fluss Flink Sink / Taskmanager
| Metric | Before | After |
Delta |
|------------------------------------------------|----------:|----------:|-------:|
| Writer records/s, avg (sum across 16 subtasks) | 2,526,516 | 2,826,183 |
+11.9% |
| Writer records/s, peak | 3,342,630 | 4,421,649 |
+32.3% |
| Writer records/s, p95 | 3,082,199 | 4,198,872 |
+36.2% |
| TaskManager JVM CPU load, avg | 51.5% | 44.1% |
-7.4% |
| JVM GC time, ms per second, avg | 1,433 | 999 |
-30.3% |
| Writer task busy time, ms/s, avg | 642.5 | 449.7 |
-30.0% |
| Writer task idle time, ms/s, avg | 357.5 | 550.3 |
+53.9% |
| Records per batch, avg | 40,971 | 53,531 |
+30.7% |
| Bytes per batch, avg | 1,267,516 | 1,645,327 |
+29.8% |
#### Per-record CPU cost
TaskManager level: `cpu_load / throughput`
- Before: 0.515 / 2,526,516 = 204 ns of CPU per record
- After: 0.441 / 2,826,183 = 156 ns of CPU per record
- Down 23.5%
Writer-task level: `busy_ms_per_s / throughput`
- Before: 642.5 / 2,526,516 = 254 ns per record
- After: 449.7 / 2,826,183 = 159 ns per record
- Down 37.4%
The 37% drop at the writer-task level matches before flamegraph attributed
roughly half the writer CPU to
`TimeZone.getTimeZone` plus the `AutoPartitionStrategy.from` construction
around
it. Removing both shaves about that much off the per-record cost. The
remaining
work (Arrow encoding, ZSTD, serialisation) didn't change.
#### What got faster
Throughput is up at every percentile. Peak and p95 moved more than the mean,
which fits the picture of the writer no longer being the bottleneck.
CPU is down even though throughput is up. TM doing more records with less
work.
TM GC time dropped. The fix removes the per-record allocation of
`AutoPartitionStrategy` and the throwaway `Configuration` view, which is the
kind of short-lived garbage that drives young-gen churn.
Batches got about 30% bigger by both record count and bytes. With the writer
faster, the batch window naturally accumulates more records before the
timeout
fires, so each RPC carries more payload.
#### Fluss Server-side
| Metric |
Before | After | Delta |
|-------------------------------------------------------------------|-----------:|-----------:|----------------------------------------------:|
| `fluss_tabletserver_table_messagesInPerSecond`, avg |
2,526,438 | 2,816,269 | +11.5% |
| `fluss_tabletserver_table_messagesInPerSecond`, peak |
3,346,781 | 4,464,978 | +33.4% |
| `fluss_tabletserver_table_messagesInPerSecond`, p95 |
3,077,221 | 4,198,341 | +36.4% |
| `fluss_tabletserver_table_bytesInPerSecond`, avg | 77.9
MB/s | 86.6 MB/s | +11.2% |
| `fluss_tabletserver_table_bytesInPerSecond`, peak | 103.2
MB/s | 138.1 MB/s | +33.8% |
| `fluss_tabletserver_table_bytesOutPerSecond`, avg | 99.7
MB/s | 90.0 MB/s | -9.7% (see note below) |
| `fluss_tabletserver_table_totalProduceLogRequestsPerSecond`, avg |
149.1 | 186.7 | +25.2% |
| `fluss_tabletserver_table_totalProduceLogRequestsPerSecond`, peak |
427.5 | 373.2 | -12.7% (fewer peak RPCs at higher throughput) |
| `fluss_tabletserver_table_failedProduceLogRequestsPerSecond` |
0 | 0 | |
**Flink Task - Fluss Sink Flamegraph Before**
<img width="1960" height="1450" alt="Screenshot 2026-06-25 at 14 21 58"
src="https://github.com/user-attachments/assets/71d015f1-96b6-48b6-9df0-67dc1b9904bf";
/>
**Flink Task - Fluss Sink Flamegraph After**
No trace of getTimezone
<img width="2688" height="1548" alt="Screenshot 2026-06-24 at 23 24 43"
src="https://github.com/user-attachments/assets/1e9ef33a-be1f-4a3a-9f8a-9b6b9888ba93";
/>
**Flink**
Note: on left hand side is before and on right hand side is after
<img width="3422" height="1086" alt="Screenshot 2026-06-25 at 16 18 32"
src="https://github.com/user-attachments/assets/6c89415f-a19f-4a20-9b0e-78e1b07f5cbd";
/>
<img width="3422" height="666" alt="Screenshot 2026-06-25 at 16 19 05"
src="https://github.com/user-attachments/assets/3b95fde9-1dce-47ef-8708-cabcd0098688";
/>
<img width="3422" height="1134" alt="Screenshot 2026-06-25 at 16 19 32"
src="https://github.com/user-attachments/assets/d0b168fd-b498-4d6f-8c85-e985c29872f6";
/>
<img width="3422" height="600" alt="Screenshot 2026-06-25 at 16 19 51"
src="https://github.com/user-attachments/assets/44340e67-2826-43a5-b4af-b30f49c08de5";
/>
<img width="3422" height="450" alt="Screenshot 2026-06-25 at 16 20 34"
src="https://github.com/user-attachments/assets/be725055-8a59-44d9-81fe-371f0a76155d";
/>
<img width="3422" height="500" alt="Screenshot 2026-06-25 at 16 20 50"
src="https://github.com/user-attachments/assets/e888dcdb-eede-4974-9160-2c5ca7590976";
/>
**Fluss**
Note: on left hand side is before and on right hand side is after
<img width="3422" height="522" alt="Screenshot 2026-06-25 at 16 21 49"
src="https://github.com/user-attachments/assets/cb5e10eb-b2ee-43a2-a383-eab36f7eb2b0";
/>
<img width="3422" height="640" alt="Screenshot 2026-06-25 at 16 23 46"
src="https://github.com/user-attachments/assets/eeb4d19a-5e45-4794-941b-ef6a623c474e";
/>
<img width="3422" height="702" alt="Screenshot 2026-06-25 at 16 25 11"
src="https://github.com/user-attachments/assets/7bd511a6-a061-4a54-93b3-27054c44ba27";
/>
<img width="3422" height="632" alt="Screenshot 2026-06-25 at 16 27 04"
src="https://github.com/user-attachments/assets/862763f9-eaad-4c9e-90fa-d5dd87a73f3c";
/>
<img width="3422" height="702" alt="Screenshot 2026-06-25 at 16 28 22"
src="https://github.com/user-attachments/assets/88d41940-dbb4-423a-bf3b-cbb76ac0f01f";
/>
### API and Format
No API or storage format changes.
### Documentation
None needed. No new feature, no user-visible behaviour change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
