junaiddshaukat opened a new pull request, #39627: URL: https://github.com/apache/beam/pull/39627
## Summary Part of #18479. Adds the runner's user documentation and marks it experimental, which was the remaining documentation deliverable. Reviewing #39611 the point came up that the features which are core but not yet implemented must be written down; this is where they are. ## The documentation `website/www/site/content/en/documentation/runners/kafkastreams.md`, linked from the runners menu alongside the others. It covers what the runner is and why someone would pick it, how to start the job server and submit a pipeline, every pipeline option with its default, the internal topics it creates, and two lists that matter more than the rest: what is supported and what is not. The unsupported list is deliberately specific rather than a general disclaimer, and says what each gap means for a user: * side inputs; stateful `ParDo` and user timers; merging windows, so no session windows; custom `WindowFn`s; splittable `DoFn`; `TestStream` * reading a source in parallel — a source is split into exactly one part, and one that insists on splitting further is rejected rather than silently truncated * `maxBundleTimeMs` is accepted but has no effect, because closing a bundle from a wall-clock punctuator duplicated output against a real broker and the cause is not yet understood * `finalizeCheckpoint` is not called, so a source relying on finalization to acknowledge data will not see it * only attempted metrics, not committed ## Marking it experimental `KafkaStreamsRunner`'s javadoc now opens by saying the runner is experimental, naming the main gaps and warning that behaviour and options may change, with links to the documentation and the tracking issue. The documentation says the same in its own section, so someone arriving from either direction learns it before they invest in it. ## A task to start the job server The documentation needs a command that works, and there wasn't one — the job server could only be started by invoking `KafkaStreamsJobServerDriver` directly. `:runners:kafka-streams:runJobServer` now does it, with arguments passed through `-PjobServerArgs`. Verified by running it: the job service comes up on `localhost:8099` and artifact staging on `localhost:8098`, which is what the documentation tells the reader to expect. ## Testing The runner code change is javadoc only; the gate is unchanged and still green. ``` ./gradlew :runners:kafka-streams:validatesRunner # 59 tests, 0 failures ./gradlew :runners:kafka-streams:build # 86 unit tests, spotless + checker + errorprone ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
