[GitHub] [beam] Amraneze commented on pull request #25886: Fix Illegal to use destination for jms producer

[via GitHub]

Amraneze commented on PR #25886:
URL: https://github.com/apache/beam/pull/25886#issuecomment-1478220381

   @Abacn It's the same pipeline one was running with the version 2.45 + the 
new code for JmsIO and one was with the version 2.46. Both of them have the 
same configuration (same number of workers, ....) and they use Dataflow Runner 
v1 (we had some drawbacks with v2)
   
   Here is a screenshot of the port usage after running the pipeline (using 
only beam 2.46) with `numberOfWorkerHarnessThreads` = 32. It stabilizes at 2k 
open ports.
   
   <img width="1664" alt="image" 
src="https://user-images.githubusercontent.com/28459763/226658971-75509571-43a5-4880-96c2-a6e7ea999eee.png";>
   
   A screenshot of the port usage after running the pipeline (using beam 2.46 + 
the fix of this PR). It stabilizes at 2k open ports.
   
   <img width="1672" alt="image" 
src="https://user-images.githubusercontent.com/28459763/226668678-f87aca1f-dbeb-4504-909c-c94da306af19.png";>
   
   A screenshot of the port usage after running the pipeline (using beam 2.45 + 
the fix of this PR + new JmsIO code). It stabilizes at 2k open ports.
   
   <img width="1666" alt="image" 
src="https://user-images.githubusercontent.com/28459763/226680600-37804041-fd74-4359-a187-3b5ac7457aac.png";>
   
   >Note: The pipelines are running with the same configuration, `18` workers 
and `n2-highmem-4` without `autoscaling`
   
   I guess, we will need to setup the connection at the `setup cycle` instead 
of `startBundle cycle`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscr...@beam.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org