Craig, OPC ( https://issues.apache.org/jira/browse/MINIFICPP-819 ) and Modbus ( https://issues.apache.org/jira/browse/MINIFICPP-897 ) are on the way for MiNiFi c++, hopefully both will be part of next release (0.7.0). It's gonna be legen... wait for it! :)
Regards, Arpad On Wed, Jul 31, 2019 at 2:30 AM Craig Knell <[email protected]> wrote: > Hi Folks > > That's our use case now. All our Models are run in python. > Currently we send events to the ML via http, although this is not optimal > > Our use case is edge ML where we want a light weight wrapper for > Python code base. > Jython however does not work with the code base > I'm think of changing the interface to some thing like REDIS for pub/sub > Id also like this to be a push deployment via minifi > > Also support for sensors via protocols via Modbus and OPC would be great > > Craig > > On Wed, Jul 31, 2019 at 1:43 AM Joe Witt <[email protected]> wrote: > > > > Definitely something that I think would really help the community. It > > might make sense to frame/structure these APIs such that an internal > option > > could be available to reduce dependencies and get up and running but that > > also just as easily a remote implementation where the engine lives and is > > managed externally could also be supported. > > > > Thanks > > > > > > On Tue, Jul 30, 2019 at 1:40 PM Andy LoPresto <[email protected]> > wrote: > > > > > Yolanda, > > > > > > I think this sounds like a great idea and will be very useful to > > > admins/users, as well as enabling some interesting next-level > functionality > > > and insight generation. Thanks for putting this out there. > > > > > > Andy LoPresto > > > [email protected] > > > [email protected] > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > > > > > On Jul 30, 2019, at 5:55 AM, Yolanda Davis < > [email protected]> > > > wrote: > > > > > > > > Hello Everyone, > > > > > > > > I wanted to reach out to the community to discuss potentially > enhancing > > > > NiFi to include predictive analytics that can help users assess and > > > predict > > > > NiFi behavior and performance. Currently NiFi has lots of metrics > > > available > > > > for areas including jvm and flow component usage (via component > status) > > > as > > > > well as provenance data which NiFi makes available either through > the UI > > > or > > > > reporting tasks (for consumption by other systems). Past discussions > in > > > the > > > > community cite users shipping this data to applications such as > > > Prometheus, > > > > ELK stacks, or Ambari metrics for further analysis in order to > > > > capture/review performance issues, detect anomalies, and send alerts > or > > > > notifications. These systems are efficient in capturing and helping > to > > > > analyze these metrics however it requires customization work and > > > knowledge > > > > of NiFi operations to provide meaningful analytics within a flow > context. > > > > > > > > In speaking with Matt Burgess and Andy Christianson on this topic we > feel > > > > that there is an opportunity to introduce an analytics framework that > > > could > > > > provide users reasonable predictions on key performance indicators > for > > > > flows, such as back pressure and flow rate, to help administrators > > > improve > > > > operational management of NiFi clusters. This framework could offer > > > > several key features: > > > > > > > > - Provide a flexible internal analytics engine and model api which > > > > supports the addition of or enhancement to onboard models > > > > - Support integration of remote or cloud based ML models > > > > - Support both traditional and online (incremental) learning > methods > > > > - Provide support for model caching (perhaps later inclusion into > a > > > > model repository or registry) > > > > - UI enhancements to display prediction information either in > existing > > > > summary data, new data visualizations, or directly within the > > > flow/canvas > > > > (where applicable) > > > > > > > > For an initial target we thought that back pressure prediction would > be a > > > > good starting point for this initiative, given that back pressure > > > detection > > > > is a key indicator of flow performance and many of the metrics > currently > > > > available would provide enough data points to create a reasonable > > > > performing model. We have some ideas on how this could be achieved > > > however > > > > we wanted to discuss this more with the community to get thoughts > about > > > > tackling this work, especially if there are specific use cases or > other > > > > factors that should be considered. > > > > > > > > Looking forward to everyone's thoughts and input. > > > > > > > > Thanks, > > > > > > > > -yolanda > > > > > > > > -- > > > > [email protected] > > > > @YolandaMDavis > > > > > > > > > > -- > Regards > > Craig Knell > Mobile: +61 402 128 615 > Skype: craigknell >
