+1. Another thing we should look into is comments/code documentation in apex-core and apex-malhar. I think it can be significantly enhanced.
On 11/28/16, 9:53 PM, "Tushar Gosavi" <[email protected]> wrote: Do we have documents explaining internals of Apex? Would it be a good idea to add document explaining following on apex.apache.org to help newcomers understand overall execution flow of application and apex capabilities, this might also help them to add new features. - Startup of application - logical plan formation and properties injection. - copying of jars/resources - handling of old application state. - Master initialization - converting logical plan to physical plan - container assignments with locality and affinity - initial checkpoints - deployment - restarting from killed application (-originalId) - StramChild functionality - Operator deploy requests and operator lifecycle management. - handling of checkpoints - WindowGenerator - Sending hearbeats - handling of various commands from Stram. - Operator request response model. - Bufferserver management. (purge/reset/disk spooling/back-pressure handling) - Run-time activities at master. - heartbeat monitoring - stats listeners - recovery from operator failure - dynamical plan change - Tushar.
