Hi, I am Vincent from Renault Digital. We are using JmsIO connector within our organization, and we are facing the following issue:
A lot of our company dataflow jobs use JmsIO connector, as our message bus widely use AMQP protocol. JmsIO doesn't implement the getSplitBacklogBytes nor getTotalBacklogBytes methods, there is no backlog estimate, and dataflow auto scaler is not able to allocate workers depending on the backlog size (scale down and up). Therefore we don't use autoscaling, and we have a fix number of workers. I have created the following JIRA ticket: https://issues.apache.org/jira/browse/BEAM-12634 And I would like to do the following proposal to implement auto scaling feature within JmsIO: https://docs.google.com/document/d/1LMPpMpn9DByQcyCdRrTMBa-8ltByO4aqNYY4D9h3m18/edit?usp=sharing Regards, Vincent Confidential C -- Disclaimer ------------------------------------ Ce message ainsi que les eventuelles pieces jointes constituent une correspondance privee et confidentielle a l'attention exclusive du destinataire designe ci-dessus. Si vous n'etes pas le destinataire du present message ou une personne susceptible de pouvoir le lui delivrer, il vous est signifie que toute divulgation, distribution ou copie de cette transmission est strictement interdite. Si vous avez recu ce message par erreur, nous vous remercions d'en informer l'expediteur par telephone ou de lui retourner le present message, puis d'effacer immediatement ce message de votre systeme. *** This e-mail and any attachments is a confidential correspondence intended only for use of the individual or entity named above. If you are not the intended recipient or the agent responsible for delivering the message to the intended recipient, you are hereby notified that any disclosure, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender by phone or by replying this message, and then delete this message from your system.
