Hi dev, Recently, I have been working on introducing NC to NC messaging for different messages needed to be communicated between different NCs. Right now, our current NC to NC network communication is only used for job communication to send data in the Hyracks job pipeline.
I have completed the implementation of additional network communication interfaces using Netty when I was asked to check the feasibility of using the existing infrastructure to perform this task. As with every task that requires complete understanding of some existing AsterixDB / Hyracks components, I had to spend sometime reading and tracing code. During this, I started two documents: 1. Node Controller Bootstrap: In this document, I list the sequence of actions taken when the NC starts up. https://cwiki.apache.org/confluence/display/ASTERIXDB/Node+Controller+Bootstrap 2. Node Controller Network Threads: In this document, I list the different network threads and how each of them work. https://cwiki.apache.org/confluence/display/ASTERIXDB/Node+Controller+Network+Threads We still need better and more documentation of what components we have, where they live, what they do, and how they work. If you can help achieve that, then please do. At least, if you find yourself in a situation where you need to do some code reading to understand something, take the opportunity to contribute to the documentation. Hope others make good use of these, ~Abdullah.
