aliehsaeedii opened a new pull request, #23075: URL: https://github.com/apache/kafka/pull/23075
### What The smoke-test application already uses `suppress(untilWindowCloses(...))` (`SmokeTestClient#183,195`), and `streams_application_upgrade_test.py` already drives it across versions — but no system test ever set `dsl.store.format`. With the config unset, the headers-aware suppress buffer writes byte-identical V3 records, so the existing matrix passes regardless of KAFKA-20413. This adds a HEADERS-configured transition across the 4.3/trunk boundary in both directions. ### Why this boundary The headers-aware suppress buffer landed in KAFKA-20413 (#22165) on 2026-07-29. Trunk was bumped to 4.4.0-SNAPSHOT on 2026-03-19, so 4.3 has the `dsl.store.format` config (added 2026-02-27) but *not* the headers-aware buffer. The two sides therefore write the suppress changelog differently: - **4.3** writes the whole `BufferValue` into the record value. - **trunk** writes only the plain value bytes into the record value, shipping the value/timestamp/headers prefixes in extra Kafka record headers. Both tag the record as V3. Because the suppress buffer is in-memory only, every restart replays its entire changelog, so this is a genuine cross-format restore test: on upgrade, trunk must restore records carrying no value-part headers; on downgrade, 4.3 must cope with records whose prefixes it never learned to read. ### Changes - Add an `extra_configs` pass-through to `StreamsSmokeTestBaseService` and `StreamsSmokeTestJobRunnerService`, merged last in `prop_file()` so a test can override the smoke defaults. Appended as a trailing keyword argument, so existing call sites are unaffected. - Extract `_run_app_transition()` from `test_app_upgrade()` so the target version is a parameter instead of being hardcoded to `DEV_VERSION` — which is what the downgrade direction needs. `test_app_upgrade()` still passes `DEV_VERSION`, so its matrix behavior is unchanged. - Add `test_suppress_headers_app_transition`, parameterized on `direction=["upgrade", "downgrade"]`, `full` bounce only. - Assert that no instance logged a suppress-changelog restore rejection during the transition. ### On the restore assertion Since both formats tag the record V3, a mismatch can be *misread* rather than *rejected* — silent prefix loss with no exception. The existing harness only verifies that instances rejoin, process records, and close cleanly; it does not compare output topics (KAFKA-10202). The added check catches the rejection case (`"Restoring apparently invalid changelog record"`) but cannot catch silent corruption. Closing that gap needs output verification, which is KAFKA-10202's scope rather than this ticket's. `rolling` is intentionally not covered: the existing matrix only runs `bounce_type=["full"]`, so the rolling path in `do_stop_start_bounce` is not currently exercised, and this keeps the addition to two new `num_nodes=9` jobs. ### Testing - `python3 -m py_compile` passes on both changed files; `git diff --check` is clean. - The Ducktape system test has **not** been run yet — it needs a real system-test cluster. I will report the results of a full run on this PR before it should be considered ready to merge. No API, protocol, or KIP changes; test-only. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
