xinyuiscool commented on PR #23313: URL: https://github.com/apache/beam/pull/23313#issuecomment-1258745455
> Does this PR remove the limitation of `job.container.thread.pool.size > 1` and `max.bundle.size > ` existing together? > > If not, then the place where we set `setNumThreadsForProcessElement` to > 1 should throw validation exception right? To enable this feature, I used the following ways: - `numThreadsForProcessElement`: this is the user-facing SamzaPipelineOption, which can be set by typical beam users. - `job.container.thread.pool.size`: for legacy samza users, since they are familiar with this Samza config, I use this samza config to set numThreadsForProcessElement option. Afterwards the config is removed so that it won't confuse the samza engine to run the thread pool in the task. I still keep the validation of checking `job.container.thread.pool.size > 1` and `max.bundle.size > 1` there since the above logic will override the `job.container.thread.pool.size` to empty. So this condition will happen only if some user is manipulating the config after the config is generated. Does this make sense to you? We can chat offline if needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
