Hi Prabath, Agree on your points. I am +1 to have analytics separate.
--Srinath On Thu, Mar 24, 2016 at 5:06 PM, Prabath Abeysekera <[email protected]> wrote: > Hi Srinath, > > On Thu, Mar 24, 2016 at 1:48 PM, Srinath Perera <[email protected]> wrote: > >> Hi All, >> >> Prabath, if analytics is not in the device type, where should it go? >> > > IMO, we can have it as a separate project structure with a separate > packaging as part of the underlying tooling platform. Just as how things > are done when composing a typical enterprise integration scenario involving > ESB configs, DSS configs, etc, one may create a multi-module maven project > with device management plug-in skeleton as one module and analytics as > another module. When built, there will be two resultant packagings one to > be deployed into device management nodes and the other to DAS. If the same > developer is expected to take care of developing the device management > plugin implementation and the analytics story, then we can may be define a > composite maven project artifact, when clicked, will create the > aforementioned structure. > > WDYT? > > >> >> I do not agree with your observation that functional and >> non-functional parts should be separate. It helps to have it separate if >> non-functional parts can be reused with other devices. However, that is not >> the case here. >> > > IMO, it's not only reusability that defines it in this particular context. > The fact that analytics can change and evolve independently from the device > management plug-in implementation should also be considered as a valid > requirement. This somewhat aligns with the first question you've raised > below, I believe. > > The reason is, organizational analytics related requirements change over > time depending on various business strategies that they might adopt, but > the plug-in implementation might not. Also, the degree of analytics > required too would change from one organization to another even if they > deploy the same IoTS solution equipped with the same plug-in set. If we > ship analytics bundled into the plug-in, we present things to the end-user > as something tied into a plug-in imeplementation, which doesn't appear to > be correct. So, we should have a mechanism that is not coupled to a plug-in > implementation, yet functions in tandem with it, to cater that sort of > requirements. > > For instance, let's imagine there exists Organizations Org1 and Org2. They > both are interested in using IoTS, which has the capability to handle > device DX, produced by a manufacturer ManX. There's a good chance that the > analytics needed for Org1 and Org2 might be different? So, a single > solution that is shipped with built-in analytics(bundled with the plug-in) > might not be suitable for both organizations. Instead, they would want to > customize the KPIs to suit their business requirements. One might think > this is hypothetical, but I thought there's a good enough probability that > this might be the case in a real deployment. > > >> >> I think, as sumedha said, we should ship what we have and change based on >> how it is used. >> > > Anyway, if everyone thinks, we should wait for industry feedback to make a > decision on this, yeah of course, we'll do it that way. > > >> >> If we choose to ship analytics bundled, we need to answer two questions. >> >> >> 1. Would plugin developer will do all analytics or do we have use >> cases where end users will also create their own analytics? If they want >> to >> build their own analytics, we need think through how it work. >> >> To my understanding, both scenarios are possible. > > >> >> 1. There will be analytics that use data from multiple device types? >> How would such cases handled? >> >> Well, in the case of mobile devices, this certainly is a valid scenario > as people usually prefer to have an aggregated view of how devices as a > whole behave within an organization adhering compliance rules, etc. Even in > other typical IoT contexts, when it comes implementing monitoring/analytics > upon groups that have multiple device types in them i.e. a smart home > solution, you essentially need to have something similar, IMO. > > IoT experts, please do comment in and correct me if I'm wrong. > > Cheers, > Prabath > > --Srinath >> >> >> >> On Thu, Mar 24, 2016 at 12:41 PM, Prabath Abeysekera <[email protected]> >> wrote: >> >>> Hi Sumedha, >>> >>> IMO, the device management plug-in layer does show what features can be >>> controlled, what device info to be captured, etc at a somewhat satisfactory >>> level already. If this is not sufficient, we need to think further and see >>> what needs to be added at a functional level. >>> >>> Also, I agree that there's no "huge" problem in having things bundled >>> together. But, IMO, I feel that it's more logical to separate out >>> functional bits and non-functional bits, that will not only make a clear >>> demarcation between the two aspects, but also that will make the >>> implementation as well as the positioning of the implementation more >>> cleaner and cleanly presentable. >>> >>> Referring to the example you've brought in, which is around bringing up >>> an analytics story upon the Greenhouse story, that's exactly what I tried >>> to highlight too. Users would more often look at integrated analytics, >>> particularly when it comes to real world systems, so just "plug-in >>> contained" analytics will not be sufficient in majority of the cases, IMO. >>> So, we need to think beyond that level and propose and approach that will >>> be a better fit for majority of the real world use-cases. >>> >>> I totally agree on the fact that we just started getting things rolling. >>> In fact, no one as yet seems to entirely know what exactly is IoT or how >>> things will evolve in the future. As you have said, we can surely wait and >>> see how the industry responds to things and then make things evolve at our >>> end. But do we really have to rely on the industry feedback all the time to >>> do what appears to be logical? :) >>> >>> Also, the intention of this mail is not about making changes to the >>> current approach right at the last min causing possible delays in proposed >>> release timelines. Rather this is something that I thought would be logical >>> to be added to the product maybe in a future release. >>> >>> Cheers, >>> Prabath >>> >>> On Thu, Mar 24, 2016 at 11:22 AM, Sumedha Rubasinghe <[email protected]> >>> wrote: >>> >>>> My take on this is slightly different. >>>> >>>> First @ a conceptual level when we say IoT Server can be extended by a >>>> plugin, we should have a clear definition of that plugin to have X, Y, Z >>>> capabilities. So I feel having analytics @ device type definition level >>>> (conceptually) is not a problem. >>>> >>>> Going by that thinking, I don't see a huge problem in **first version >>>> of Maven archetype for creating device type plugin** having template >>>> analytics scripts generated. >>>> >>>> (For anyone interested instructions are available @ >>>> https://docs.wso2.com/display/IoTS100/Creating+a+New+Device+Plugin+via+the+Maven+Archetype >>>> More detailed documentation available @ >>>> https://docs.google.com/document/d/1ms8xJInbQlSt213Rh_cju7Amt27dLgf8twYWVDQPZ3E/edit >>>> ) >>>> >>>> >>>> On the other hand, I feel the level of analytics a device type plugin >>>> can have is very limited compared to a real life view of a devices >>>> deployment. >>>> >>>> Let me take an example. >>>> >>>> A Greenhouse has an air circulation system, humidity control & water >>>> sprinklers. As per our IoT Server, these are instances of different device >>>> types. Having per device analytics would not full fill the need of >>>> Greenhouse operator. He should rather see an integrated analytics view of >>>> all these subsystems to make decisions and control. >>>> >>>> Now where do we support this? Deploying a separate analytics bundle can >>>> be a solution. But that cannot live on it's own without having a reference >>>> to device types & respective even streams. So it's not a simple solution. >>>> >>>> In a nutshell, I am trying to say, >>>> - We have not even touched the tip of the iceberge yet >>>> - There is a long way for us to go to get to the real level >>>> >>>> - Rather than trying to justify now about the best approach, let's ship >>>> what we have now and evolve based on industry feedback. Cos, IMO what we >>>> have is not 100% fail proof or completely bound to fail. >>>> >>>> >>>> On Wed, Mar 23, 2016 at 10:37 PM, Ruwan Yatawara <[email protected]> >>>> wrote: >>>> >>>>> Hi Prabath, >>>>> >>>>> IoTS is basically the representation of WSO2's IoT Platform vision in >>>>> the flesh (Well, not literally). The idea behind it, is to provide users >>>>> with the necessary tools and components needed to get started with writing >>>>> device types that will later go in actual deployments. So someone who is >>>>> evaluating or test driving our platform per say, would find a more >>>>> integrated and therefore appealing story by discovering our device >>>>> management capabilities and analytics capabilities, integrate seamlessly >>>>> with no additional effort. >>>>> >>>>> Analytics maybe a non-functional requirement when we try to view IoTS >>>>> as just a device management framework. But in reality it is much bigger >>>>> than that, the whole driving force behind IIOT is the need to improve >>>>> operational efficiency of businesses through monitoring and analysis of >>>>> data from an array of sensory networks. Therefore IMHO, by all means >>>>> analytics becomes a primary functional requirement. We are just getting >>>>> our >>>>> beaks wet by making available a bunch of DAS scripts with the plugin. In >>>>> the future, we may even find it necessary to put data models for machine >>>>> learning frameworks in there. >>>>> >>>>> Furthermore, having all the information pertaining to a particular >>>>> device type maybe advantageous when it comes to import/export of device >>>>> types. For an example, lets for a moment assume there is a device cloud by >>>>> the name of abc.com that runs on top of wso2's IOT framework. I as an >>>>> independent hardware vendor have written a device type to manage and >>>>> monitor my device on my laptop using the tooling available for IoTS. So >>>>> with the current approach, all I have to do is grab that device type >>>>> (Which >>>>> should ideally be a deploy-able archive, contrary to our current feature >>>>> based approach) and upload it to abc.com cloud. In one go, my device >>>>> type will be deployed. >>>>> >>>>> Yes, this does bring to the table, the need to deploy various types of >>>>> different artifacts in different systems. But I believe we have done >>>>> something similar when it comes to deployment of imported API artifacts in >>>>> APIM (Which are deploy-able zip archives, containing sequences, images, DB >>>>> entries and what-not) >>>>> >>>>> Due to reasons mentioned above. I am of the opinion that we should >>>>> bundle analytics scripts with the plugin. >>>>> >>>>> Thanks and Regards, >>>>> >>>>> Ruwan Yatawara >>>>> >>>>> Senior Software Engineer, >>>>> WSO2 Inc. >>>>> >>>>> email : [email protected] >>>>> mobile : +94 77 9110413 >>>>> blog : http://ruwansrants.blogspot.com/ >>>>> www: :http://wso2.com >>>>> >>>>> >>>>> On Wed, Mar 23, 2016 at 5:54 PM, Prabath Abeysekera <[email protected] >>>>> > wrote: >>>>> >>>>>> Hi Everyone, >>>>>> >>>>>> As most of you know, IoTS is a unified platform that allows users to >>>>>> come up with "plugins" and seamlessly add them to the framework to be >>>>>> able >>>>>> to manage different types of devices. For instance, if an organization >>>>>> named "Org-X" manufactures a device type "Dev-X", they can simply adopt >>>>>> IoTS and extend its plug-in model to come up with a plug-in to support >>>>>> end-to-end device management and more upon "Dev-X" devices. To make >>>>>> plug-in >>>>>> generation process easier for the developers who are involved in the >>>>>> same, >>>>>> a maven archetype is used. The aforesaid archetype is simply used to >>>>>> create >>>>>> a skeleton of the plug-in, which can then be enriched with different >>>>>> communication protocols, etc depending on the expectations of different >>>>>> device manufacturers, etc. >>>>>> >>>>>> Currently, the maven archetype explained above has the following >>>>>> structure. >>>>>> >>>>>> -- component >>>>>> --*analytics* >>>>>> --controller >>>>>> --manager >>>>>> --plugin >>>>>> --ui >>>>>> >>>>>> Once the above skeleton is generated, then the developers customize >>>>>> the same to come up with the desired plug-in implementations and package >>>>>> things up to be able to push it to the CDM-F. >>>>>> >>>>>> Right now the model being used is that, all the aforementioned >>>>>> resources such as the plugin implementation, analytics scripts, etc are >>>>>> packed up as a single archive. IMO, having analytics scripts bundled up >>>>>> with the rest of the bits is not a good approach depending on a few >>>>>> reasons. >>>>>> >>>>>> 1. When it comes real production deployments, these analytics scripts >>>>>> will be deployed in a separate DAS cluster as opposed to how things are >>>>>> when the same is done within the scope of a single server. So, if the two >>>>>> components are running separately, there's no point packaging everything >>>>>> up >>>>>> as a single archive as we'd have to break things down to two separate >>>>>> packagings (a) to be deployed into IoTS (b) to be deployed into DAS. >>>>>> >>>>>> 2. Analytics, I believe, primarily is a non-functional requirement. >>>>>> Therefore, bundling non-functional bits with functional bits is not a >>>>>> good >>>>>> practice. >>>>>> >>>>>> 3. If we ship analytics scripts with a particular plug-in, the >>>>>> natural expectation is that it would only cover specific bits bound to >>>>>> the >>>>>> scope of particular plug-in. However, in a practical environment, what >>>>>> majority of the folks are looking at are much more advanced and composite >>>>>> KPIs and related analytics to be able to help organizational strategies, >>>>>> etc. So it is highly likely that people might generally be interested in >>>>>> brining in analytics as a separate step relieving the need to pack the >>>>>> scripts, etc with the functional bits. i.e. plug-in implementation. >>>>>> >>>>>> >>>>>> Instead what I propose is to have a separate "project type" or >>>>>> something similar as part of the underlying tooling platform, which can >>>>>> effectively be used to compose all analytics related scripts, etc. We can >>>>>> then package it up separately and deploy into the relevant DAS clusters >>>>>> to >>>>>> get the analytics up and running. >>>>>> >>>>>> >>>>>> Some of the stuff I've brought up might be subjective, but I am of >>>>>> the opinion that we need to remove analytics related scripts from the >>>>>> plug-in skeleton. >>>>>> >>>>>> WDYT? >>>>>> >>>>>> >>>>>> Cheers, >>>>>> Prabath >>>>>> -- >>>>>> Prabath Abeysekara >>>>>> Technical Lead >>>>>> WSO2 Inc. >>>>>> Email: [email protected] >>>>>> Mobile: +94774171471 >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> [email protected] >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> /sumedha >>>> m: +94 773017743 >>>> b : bit.ly/sumedha >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> Prabath Abeysekara >>> Technical Lead >>> WSO2 Inc. >>> Email: [email protected] >>> Mobile: +94774171471 >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> ============================ >> Srinath Perera, Ph.D. >> http://people.apache.org/~hemapani/ >> http://srinathsview.blogspot.com/ >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Prabath Abeysekara > Technical Lead > WSO2 Inc. > Email: [email protected] > Mobile: +94774171471 > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- ============================ Blog: http://srinathsview.blogspot.com twitter:@srinath_perera Site: http://home.apache.org/~hemapani/ Photos: http://www.flickr.com/photos/hemapani/ Phone: 0772360902
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
