mattcasters opened a new pull request, #8682: URL: https://github.com/apache/hop/pull/8682
Specify copies and a transform partition schema start threads and route rows only on the local Hop pipeline engine. On Beam, including Flink, and on native Spark, the runner chooses parallelism. The copy count and partition schema do not pin rows to the same Flink task or Spark partition. The Specify copies and Partitioning pages now say that. The same limit is noted on the pipeline overview, the partition schema metadata page, the Beam getting-started page, the Flink engine page, and the native Spark getting-started table. Beam still treats the copies field as the existing `SINGLE_BEAM` and `BATCH` flags, and Generate rows still passes a numeric copy count to Beam as the synthetic source's initial bundle count. A remote or load-balancing run configuration applies copies and partitioning only when the Hop Server executes the pipeline with the local engine. This is a documentation-only change. No Maven tests were run, and Spotless was not applied because no Java module changed. fixes #3478 ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Run `mvn clean install apache-rat:check` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [x] If you have a group of commits related to the same change, please squash your commits into one and force push your branch using `git rebase -i`. - [x] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. To make clear that you license your contribution under the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0) you have to acknowledge this by using the following check-box. - [x] I hereby declare this contribution to be licensed under the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0) - [ ] In any other case, please file an [Apache Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
