Hi all,
This is to give a heads up on the enhancement which the IoT-Analytics team
is doing on Template Manager (We've renamed the Execution Manager to
Template Manager now, to better reflect its capabilities).
Usecase:
When it comes to Analytics for IoT, it is very common that users will want
to analyse statistics such as average, min, max etc., with regards to some
device, graphically (using suitable charts).
For example, let's say a user wants to analyse the average and max values
being read by a set of temperature sensors for the past hour. He wishes to
view those analytics graphically.
With the new enhancement made to Template Manager,
this user will be able to get the required graphs generated by the Template
Manager; he will only need to map the Temperature Event Stream into a
predefined Event Stream in IoT Analytics Server (also depending on the
Templates configured behind the scene, he may be asked to specify the time
duration to monitor stats for: 1 hour, as per the given example)
Limitation in existing Template Manager:
When the user provides his input, such as the Event Stream Mapping and time
duration to monitor stats for,
the pre-configured Template, behind the scene, needs to prepare the
required Siddhi Execution Plans, Spark Scripts, Event Persistence
Configurations, Gadgets etc and deploy those.
The existing Template Manager can deploy only one artifact, for a given
analytic scenario. The artifact can be either an Execution Plan or a Spark
Script.
It cannot deploy all the artifacts required for the scenario.
Due to this reason, we have enhanced the Template Manager so that it will
be able to deploy multiple artifacts, for a given analytic scenario.
New Domain Template:
The structure of the new Domain Template is as follows (I have removed
artifacts such as Execution Plans for clarity. A complete Template can be
found at [1]):
1. <domain name="SpatialAnalyticsDomain">
2. <description>Domain for sensor data analysis</description>
3. <scenarios>
4. <scenario type="SensorAnalytics">
5. <description>Configure a sensor analytics scenario to
display statistics for a given stream of your choice</description>
6. <templates>
7. <!--Note: These will be deployed in the order they
appear here-->
8. <template type="eventstream">
9. <!--A Stream Definition Template here-->
10. </template>
11. <template type="realtime">
12. <!--A Execution Plan Template here-->
13. </template>
14. <template type="realtime"> <!--Can define
multiple Templates of same type. E.g. of type realtime, as shown here.-->
15. <!--A Execution Plan Template here-->
16. </template>
17. <template type="batch">
18. <!--A Spark Script Template here-->
19. </template>
20. <template type="gadget">
21. <!--A Gedget Template here-->
22. </template>
23. <template type="dashboard">
24. <!--A Dashboard Template here-->
25. </template>
26. </templates>
27. <streamMappings>
28. <streamMapping to=
"org.wso2.event.$sensorType.stream:1.0.0"/>
29. </streamMappings>
30. <parameters>
31. <parameter name="timeInMins" type="int">
32. <displayName>Time(Mins)</displayName>
33. <description> The sliding time period for which
the window should hold events</description>
34. <defaultValue>1</defaultValue>
35. </parameter>
36. <parameter name="sensorType" type="string">
37. <displayName>Sensor Type Name</displayName>
38. <description>The name of the sensor type
</description>
39. <defaultValue>Temperature</defaultValue>
40. </parameter>
41. </parameters>
42. </scenario>
43. </scenarios>
44. <commonArtifacts>
45. <artifact type="eventstream"> <!--This Stream can be used
across the whole Domain.-->
46. {
47. "name": "commonStream",
48. "version": "1.0.0",
49. "nickName": "",
50. "description": "",
51. "payloadData": [
52. {
53. "name": "timestamp",
54. "type": "LONG"
55. },
56. {
57. "name": "value",
58. "type": "DOUBLE"
59. }
60. ]
61. }
62. </artifact>
63. </commonArtifacts>
64. </domain>
*scenario: *
A scenario user wants to analyse. For example: analyzing average and max
Temperature for the past hour.
*domain:*
A logical collection of Scenarios. Template creator can group related
scenarios together into one Domain for better organization.
*template:*
A templated artifact.
For example, consider following Event Stream Template:
1. <template type="eventstream">
2. {
3. "name":"org.wso2.event.$sensorType.stream",
4. "version":"1.0.0",
5. "nickName":"",
6. "description":"",
7. "payloadData":[
8. {
9. "name":"sensor_id",
10. "type":"STRING"
11. },
12. {
13. "name":"sensor_value",
14. "type":"DOUBLE"
15. }
16. ]
17. }
18. </template>
At the runtime, when the user specifies an input to 'sensorType' *parameter*,
'$sensorType' in the template will be replaced with the user-input and
Event Stream config will be deployed.
For now, supported artifact types are 'eventstream', 'realtime', 'batch' ,
'eventsink', 'gadget', 'dashboard'
*streamMappings:*
This creates the link between user-defined Event Stream and the Event
Streams defined in the Template.
When Stream Mapping elements are present, users will be asked to map an
Input Event Stream(s) of his choice (for example, the Temperature Stream,
described in the usecase section above) to an Event Stream(s) defined in
the Domain Template.
*parameters:*
Parameters which the user needs to provide at the runtime.
*commonArtifacts:*
Artifacts (which are *not* templated) which are common to all the scenarios
can be put in this section, so they will only be deployed once and can be
shared thereafter, among the scenarios.
Any artifact type which is supported as a template type can be put under
commonArtifacts (so the supported types as at now would be 'eventstream',
'realtime', 'batch' , 'eventsink', 'gadget', 'dashboard')
Extensibility:
To support a new Template type, create a new carbon component, implementing
'TemplateDeployer' interface [2]. For an example, refer Event Processor
Template Deployer component [3].
Your feedback is appreciated.
Thank you
[1]
https://github.com/wso2/analytics-iots/blob/384d2ef23aa7b912a47b0b03d5f4e3818ab6e403/product/distribution/src/repository/conf/execution-manager/domain-template/SensorDataAnalyticsDomain.xml
[2]
https://github.com/wso2/carbon-analytics-common/blob/master/components/execution-manager/org.wso2.carbon.event.execution.manager.core/src/main/java/org/wso2/carbon/event/execution/manager/core/TemplateDeployer.java
[3]
https://github.com/wso2/carbon-event-processing/tree/master/components/event-processor/org.wso2.carbon.event.processor.template.deployer
--
*Dilini Muthumala*
Senior Software Engineer,
WSO2 Inc.
*E-mail :* [email protected]
*Mobile: *+94 713-400-029
_______________________________________________
Architecture mailing list
[email protected]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
