Hello Druid developers, what do you think about the future of Druid & machine learning?
Druid has been great at complex aggregations. Could (should?) It make inroads into ML? Perhaps aggregators which apply the rows against some pre-trained model and summarize results. Should model training stay completely external to Druid, or it could be incorporated into Druid's data lifecycle on a conceptual level, such as a recurring "indexing" task which stores the result (the model) in Druid's deep storage, the model automatically loaded on historical nodes as needed (just like segments) and certain aggregators pick up the latest model? Does this make any sense? In what cases Druid & ML will and will not work well together, and ML should stay a Spark's prerogative? I would be very interested to hear any thoughts on the topic, vague ideas and questions.
