Hi, Thank you Oleg and Jesse for providing me useful details. I will try to get started and get back to you soon with my ideas.
Regards, Pham Vu Tuan On Wednesday, February 14, 2018 at 4:23:13 PM UTC+7, Oleg Nenashev wrote: > > Thanks for the interest! > I will try to provide a detailed writeup about the project. > > Generally, the idea is to replace current communication layer by a pure > message bus implementation. Below you can find some things from the top of > my head: > > - All communications between master and agent happen over the message > bus > - Jenkins agents and masters can connect to a message bus. > - The topology is TBD, it may be either embedded message bus > (Jenkins master as a host) or an external service. The latter one is > preferable IMHO > - Jenkins agents query connection metadata over the bus. > - Current implementations query metadata from an HTTP/HTTPs > endpoint before connection. Getting rid of that would help > - Command transport for message bus is implemented (as Jesse > mentioned). > - It would include transparent support of command invocation, RMI, > classloading and data streaming > - We utilize message bus features to make connections over > Remoting more stable > - Fault tolerance > - The messaging layer survives short network outages > - Nice2Have: We use multiple message bus hosts so they do not > become a bottleneck > - Traffic/Request prioritization > - Currently we have an issue with logging/file transfer > messaging overloading the channel and blocking system commands. We > could > use message bus features to prevent that > - TBD: More efficient data streaming > - Implementation: > - Executable for the agent side > - Jenkins plugin, which provides the master-side logic and serves > the agent executable > - We have a reference implementation for demos > - E.g: a set of Docker images, which bring up a Jenkins cluster with a > master and several agents (e.g. with Kubernetes or Docker Compose) > - It may be somehow aligned with other ongoing activities (e.g. the > demo could be built on the top of EverGreen distro) > <https://github.com/jenkinsci/jep/tree/master/jep/300> > - TBD: a PoC for Master <=> Master communication > - In Jenkins project we have some master => master plugins like > Parameterized Remote Trigger. We could try to use a message bus to > deliver > messages between masters > - TBD: Streaming agent logs to a log storage over the message bus > - TBD: Remoting autoupdate over message bus > - Message bust engine could be a bootstrap library, which loads > Remoting library from the master after establishing the connection. > - In Jenkins project we often see issues with update Remoting > versions on the agent side. Although it would be great to solve the > issue > on the Remoting side, usage of such bootstrap could be a quick win > > Best regards, > Oleg > > вторник, 13 февраля 2018 г., 16:53:00 UTC+1 пользователь Jesse Glick > написал: >> >> On Tue, Feb 13, 2018 at 9:49 AM, Pham Vu Tuan <phamvu...@gmail.com> >> wrote: >> > I am quite interested in project >> > "Jenkins Remoting over Message Bus/Queue" and want to understand more >> about >> > the project. >> >> Probably Oleg will give you some depth but to get started >> understanding the background, look at subclasses of: >> >> >> https://github.com/jenkinsci/remoting/blob/master/src/main/java/hudson/remoting/CommandTransport.java >> >> > -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/394e00b8-5516-4e92-90c0-2fd5e01ab5ae%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
