Hi all,
This is to add more context to the stream mapping functionality with
reference to the above mail,
As mentioned above, stream mapping creates the link between the
user-defined Event Streams and the Event Streams defined in the Template.
With the improvements done to the Template Manager, users will be asked to
map an Input Event Stream(s) of their choice to an Event Stream(s) defined
in the template file. (Stream mapping will be enabled only if '
*streamMappings*' element is present in the template).
For example, given a scenario of where the user is having a raw data stream
with multiple types of sensor data types (i.e temperature data, humidity
data etc.), he can map his raw data stream to the
"org.wso2.event.$sensorType.stream:1.0.0" which is defined in the template
file as below.
*<streamMappings>*
*<streamMapping to*="org.wso2.event.$sensorType.stream:1.0.0"/>
*</streamMappings>*
Therefore the stream mapping would be from "org.wso2.raw.data.stream:1.0.0"
to "org.wso2.event.$sensorType.stream:1.0.0" and the users will have the
ability to map user defined event stream attributes to the templated
attributes. Please find the screenshot of a sample stream mapping attached
herewith (note that $sensorType has been replaced by 'Temperature' for this
scenario).
Currently this supports multiple stream mappings where the user can have
several input streams where they can be mapped into predefined streams and
these predefined streams can be easily included in the template file by
defining multiple '*streamMapping to*' elements.
For further reference, please find a sample template file with stream
mapping enabled in [1].
Your feedback is highly appreciated.
Thanks.
Thilini
[1]
https://github.com/wso2/analytics-iots/blob/384d2ef23aa7b912a47b0b03d5f4e3818ab6e403/product/distribution/src/repository/conf/execution-manager/domain-template/SensorDataAnalyticsDomain.xml
On Fri, Jun 10, 2016 at 3:39 PM, Dilini Muthumala <[email protected]> wrote:
> Hi all,
>
> This is to give a heads up on the enhancement which the IoT-Analytics team
> is doing on Template Manager (We've renamed the Execution Manager to
> Template Manager now, to better reflect its capabilities).
>
>
> Usecase:
>
> When it comes to Analytics for IoT, it is very common that users will want
> to analyse statistics such as average, min, max etc., with regards to some
> device, graphically (using suitable charts).
>
> For example, let's say a user wants to analyse the average and max values
> being read by a set of temperature sensors for the past hour. He wishes to
> view those analytics graphically.
>
> With the new enhancement made to Template Manager,
> this user will be able to get the required graphs generated by the
> Template Manager; he will only need to map the Temperature Event Stream
> into a predefined Event Stream in IoT Analytics Server (also depending on
> the Templates configured behind the scene, he may be asked to specify the
> time duration to monitor stats for: 1 hour, as per the given example)
>
>
> Limitation in existing Template Manager:
>
> When the user provides his input, such as the Event Stream Mapping and
> time duration to monitor stats for,
> the pre-configured Template, behind the scene, needs to prepare the
> required Siddhi Execution Plans, Spark Scripts, Event Persistence
> Configurations, Gadgets etc and deploy those.
>
> The existing Template Manager can deploy only one artifact, for a given
> analytic scenario. The artifact can be either an Execution Plan or a Spark
> Script.
> It cannot deploy all the artifacts required for the scenario.
>
> Due to this reason, we have enhanced the Template Manager so that it will
> be able to deploy multiple artifacts, for a given analytic scenario.
>
>
> New Domain Template:
>
> The structure of the new Domain Template is as follows (I have removed
> artifacts such as Execution Plans for clarity. A complete Template can be
> found at [1]):
>
>
> 1. <domain name="SpatialAnalyticsDomain">
> 2. <description>Domain for sensor data analysis</description>
> 3. <scenarios>
> 4. <scenario type="SensorAnalytics">
> 5. <description>Configure a sensor analytics scenario to
> display statistics for a given stream of your choice</description>
> 6. <templates>
> 7. <!--Note: These will be deployed in the order they
> appear here-->
> 8. <template type="eventstream">
> 9. <!--A Stream Definition Template here-->
> 10. </template>
> 11. <template type="realtime">
> 12. <!--A Execution Plan Template here-->
> 13. </template>
> 14. <template type="realtime"> <!--Can define
> multiple Templates of same type. E.g. of type realtime, as shown here.-->
> 15. <!--A Execution Plan Template here-->
> 16. </template>
> 17. <template type="batch">
> 18. <!--A Spark Script Template here-->
> 19. </template>
> 20. <template type="gadget">
> 21. <!--A Gedget Template here-->
> 22. </template>
> 23. <template type="dashboard">
> 24. <!--A Dashboard Template here-->
> 25. </template>
> 26. </templates>
> 27. <streamMappings>
> 28. <streamMapping to=
> "org.wso2.event.$sensorType.stream:1.0.0"/>
> 29. </streamMappings>
> 30. <parameters>
> 31. <parameter name="timeInMins" type="int">
> 32. <displayName>Time(Mins)</displayName>
> 33. <description> The sliding time period for
> which the window should hold events</description>
> 34. <defaultValue>1</defaultValue>
> 35. </parameter>
> 36. <parameter name="sensorType" type="string">
> 37. <displayName>Sensor Type Name</displayName>
> 38. <description>The name of the sensor type
> </description>
> 39. <defaultValue>Temperature</defaultValue>
> 40. </parameter>
> 41. </parameters>
> 42. </scenario>
> 43. </scenarios>
> 44. <commonArtifacts>
> 45. <artifact type="eventstream"> <!--This Stream can be
> used across the whole Domain.-->
> 46. {
> 47. "name": "commonStream",
> 48. "version": "1.0.0",
> 49. "nickName": "",
> 50. "description": "",
> 51. "payloadData": [
> 52. {
> 53. "name": "timestamp",
> 54. "type": "LONG"
> 55. },
> 56. {
> 57. "name": "value",
> 58. "type": "DOUBLE"
> 59. }
> 60. ]
> 61. }
> 62. </artifact>
> 63. </commonArtifacts>
> 64. </domain>
>
>
> *scenario: *
> A scenario user wants to analyse. For example: analyzing average and max
> Temperature for the past hour.
>
> *domain:*
> A logical collection of Scenarios. Template creator can group related
> scenarios together into one Domain for better organization.
>
> *template:*
> A templated artifact.
>
> For example, consider following Event Stream Template:
>
>
> 1. <template type="eventstream">
> 2. {
> 3. "name":"org.wso2.event.$sensorType.stream",
> 4. "version":"1.0.0",
> 5. "nickName":"",
> 6. "description":"",
> 7. "payloadData":[
> 8. {
> 9. "name":"sensor_id",
> 10. "type":"STRING"
> 11. },
> 12. {
> 13. "name":"sensor_value",
> 14. "type":"DOUBLE"
> 15. }
> 16. ]
> 17. }
> 18. </template>
>
>
> At the runtime, when the user specifies an input to 'sensorType'
> *parameter*, '$sensorType' in the template will be replaced with the
> user-input and Event Stream config will be deployed.
>
> For now, supported artifact types are 'eventstream', 'realtime', 'batch' ,
> 'eventsink', 'gadget', 'dashboard'
>
> *streamMappings:*
> This creates the link between user-defined Event Stream and the Event
> Streams defined in the Template.
> When Stream Mapping elements are present, users will be asked to map an
> Input Event Stream(s) of his choice (for example, the Temperature Stream,
> described in the usecase section above) to an Event Stream(s) defined in
> the Domain Template.
>
> *parameters:*
> Parameters which the user needs to provide at the runtime.
>
> *commonArtifacts:*
> Artifacts (which are *not* templated) which are common to all the
> scenarios can be put in this section, so they will only be deployed once
> and can be shared thereafter, among the scenarios.
> Any artifact type which is supported as a template type can be put under
> commonArtifacts (so the supported types as at now would be 'eventstream',
> 'realtime', 'batch' , 'eventsink', 'gadget', 'dashboard')
>
>
> Extensibility:
>
> To support a new Template type, create a new carbon component,
> implementing 'TemplateDeployer' interface [2]. For an example, refer
> Event Processor Template Deployer component [3].
>
>
> Your feedback is appreciated.
>
> Thank you
>
> [1]
> https://github.com/wso2/analytics-iots/blob/384d2ef23aa7b912a47b0b03d5f4e3818ab6e403/product/distribution/src/repository/conf/execution-manager/domain-template/SensorDataAnalyticsDomain.xml
> [2]
> https://github.com/wso2/carbon-analytics-common/blob/master/components/execution-manager/org.wso2.carbon.event.execution.manager.core/src/main/java/org/wso2/carbon/event/execution/manager/core/TemplateDeployer.java
> [3]
> https://github.com/wso2/carbon-event-processing/tree/master/components/event-processor/org.wso2.carbon.event.processor.template.deployer
>
> --
> *Dilini Muthumala*
> Senior Software Engineer,
> WSO2 Inc.
>
> *E-mail :* [email protected]
> *Mobile: *+94 713-400-029
>
--
*Thilini Anoratna*
Software Engineer
WSO2 Inc.; http://wso2.com
lean.enterprise.middleware
mobile: +94 778955654 <+94+778955654>
* <http://lk.linkedin.com/in/thilinianoratna>*
_______________________________________________
Architecture mailing list
[email protected]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
