I’ve been thinking about Streaming (continuous input) and incremental coccurrence.
As interactions stream in from the user it it fairly simple to use something like Spark streaming to maintain a moving time window for all input, and an update frequency that recalcs all input currently in the time window. I’ve done this with the current cooccurrence code but though streaming, this is not incremental. The current data flow goes from interaction input to geometry and user dictionary reconciliation to A’A, A’B etc. After the multiply the resulting cooccurrence matrices are LLR weighted/filtered/down-sampled. Incremental can mean all sorts of things and may imply different trade-offs. Did you have anything specific in mind?
