ctrlaltdilj commented on PR #27026:
URL: https://github.com/apache/flink/pull/27026#issuecomment-5099753375
All testing was done with Minio:
**Write (10.28 GB, 5 M × 2 KB records, unthrottled FileSink, 64 MB roll):**
old 17,886 ms (16,838–18,140), 64.95 CPU-s · new 16,574 ms
(16,310–16,719), 56.42 CPU-s · both sides: exactly 156 MPUs / 1,994 uploaded
parts / 156 completes per run
**Read (same fixed 10.28 GB / 156-object dataset, both sides read
identical objects):**
old 157,717 ms / 310.2 CPU-s · new 160,930 ms / 306.5 CPU-s · both: 156
GetObject (one per object, sequential), 314 HeadObject, 6 ListObjectsV2.
**Checkpoint latency (StateMachine-style, 90 checkpoints/run, small
state):**
s3p control p50 2–3 ms both sides; s3a p50 12–13 ms (old) vs 10.5–14 ms
(new); savepoint ~1.0–1.1 s; restore-to-first-checkpoint ~4.1–4.3 s — parity
everywhere, presto control unchanged as expected (same SDK v1 on both sides).
**Checkpoint throughput (~560 MB full hashmap snapshots every 5 s, stats
over all >400 MB checkpoints):**
s3a old: median 816 ms → 693 MB/s, 78–85 CPU-s, plus 607–761 bulk-delete
marker calls/run · s3a new: median 750 ms → **759 MB/s**, 54–57 CPU-s, 0
bulk-deletes · s3p control: 298 vs 282 MB/s (parity). The hadoop
path uses 64 MB parts (386–400 parts/run) vs presto's ~5 MB parts
(3,300–3,900/run) on both sides.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
