Hi all, Apart from the default JDBC provider shipped with the wizard, there are other data providers implemented specific to analytical data sources (DAS/CEP).
1. Batch data provider (polls data from tables stored in DAS) 2. Realtime data provider (subscribes to a web socket in DAS/CEP and listens to incoming events) 3. REST API data provider (currently work in progress) Apart from the providers, there's a collection of chart templates developed using VizGrammar library. At wizard's final stage, user has to select one of these chart for data rendition. Above providers and chart templates are bundled into a Carbon feature and can be found at [1]. Providers and chart templates will be copied to respective folders in portal app upon feature installation. E.g : Providers will be copied into /portal/extensions/providers, charts will be copied into /portal/extensions/chart-templates folder. [1] https://github.com/wso2/carbon-analytics-common/tree/master/features/analytics-gadget-templates Thanks, Dunith On Wed, May 25, 2016 at 12:06 PM, Tanya Madurapperuma <[email protected]> wrote: > Hi Suho and all, > > Seems like we need to do a small modification to the registerCallBackforPush > function. Even though we agreed to pass providerConfig and schema, we need > to have a callback function as well. Proposed modification is as follows. > > *provider's index.js* > registerCallBackforPush (providerConfig, schema, callback){ > //logic to fetch data > callback(); > } > > *in the generated gadget* > registerCallBackforPush(providerConfig, schema , function(providerData){ > draw(placeholder', chartConfig, schema, providerData); > }) > > Appreciate if you could provide your input. > > Thanks, > Tanya > > > > > On Fri, May 6, 2016 at 2:58 PM, Tanya Madurapperuma <[email protected]> > wrote: > >> Hi all, >> >> This is to provide an update regarding the above framework. >> >> We have currently completed a draft version of the wizard up to stage 2 >> with dynamic partial support and the source can be found at [1]. >> Please note that we have not paid any attention to fine tuning the UI yet. >> >> [1] >> https://github.com/wso2/carbon-dashboards/tree/gadget-generation-framework >> >> Thanks, >> Tanya >> >> On Tue, May 3, 2016 at 10:33 AM, Tanya Madurapperuma <[email protected]> >> wrote: >> >>> Hi all, >>> >>> We had a discussion again regarding this and please find the modified >>> notes below. >>> >>> Provider api (index.js) needs to implement the following functions. >>> >>> - validate(providerConfig) >>> - same as described before - is to validate the user input for >>> provider configuration >>> - getConfig() >>> - This will require the existing config.json and push any dynamic >>> fields that needs to be populated in the UI. If not will return the >>> config.json as it is. >>> - getMode() >>> - This will return whether the data provider supports polling or >>> pushing >>> - getSchema(providerConfig) >>> - This will take user filled in provider configuration and return >>> all the column names and all column types >>> - getData(providerConfig, schema, limit) >>> - schema - will take the full table schema returned from >>> getSchema method >>> - limit is to get the number of records required >>> - registerCallBackforPush(providerConfig, schema) >>> - used when the mode is push >>> >>> >>> Chart api (index.js) will also have a similar set of functions to >>> provider api. >>> >>> - getConfig(schema) >>> - will take the schema returned from getSchema function of provider >>> api and return the config to be populated in the chart configuration >>> UI >>> - validate(chartConfig) >>> - similar to provider api, will validate the user input for chart >>> configuration >>> - isProviderRequired() >>> - To be used provider configs need to be skipped in the generation >>> framework >>> - draw (chartConfig, schema, data) >>> - Return the gadget content >>> - update(data) >>> >>> @ Manu : +1 for fetching from FS by default and to have ignore list in >>> the config. >>> >>> Thanks, >>> Tanya >>> >>> >>> On Mon, May 2, 2016 at 10:06 AM, Tanya Madurapperuma <[email protected]> >>> wrote: >>> >>>> Hi Dunith, >>>> >>>> On Mon, May 2, 2016 at 6:54 AM, Dunith Dhanushka <[email protected]> >>>> wrote: >>>> >>>>> Hi Tanya, >>>>> >>>>> I too have few comments. >>>>> >>>>> 1. Can we add data access mode parameter to the provider config as >>>>> well? So there can be a parameter named "mode" which will have either >>>>> "push" or "pull" >>>>> >>>> IMO gadget creator doesn't need the flexibility to decide the mode, >>>> hence we don't need to provide mode as a provider config element. Rather we >>>> can have a method in the index.js of provider as getMode(). >>>> But provider can add the poll interval to the provider config if he >>>> allows user to configure it. >>>> >>>> >>>>> 2. Like Suhu mentioned, we can derive the list of providers by >>>>> iterating over specific FS directory. >>>>> >>>> Please see my comment for Suho's mail. >>>> >>>>> 3. Can we merge wizard steps 3 and 4 together? Then user may instantly >>>>> see the preview of a chart whenever he makes some chart configuration. >>>>> >>>> +1 >>>> >>>>> >>>>> 4. Like Suho mentioned, we'll have to pass the schemaPropertyList to >>>>> the chats index.js and generate the config.json. >>>>> >>>> +1 >>>> >>>>> >>>>> Regards, >>>>> Dunith >>>>> >>>>> Thanks, >>>> Tanya >>>> >>>>> >>>>> On Fri, Apr 29, 2016 at 9:42 PM, Sriskandarajah Suhothayan < >>>>> [email protected]> wrote: >>>>> >>>>>> Please see the comments inline. >>>>>> >>>>>> On Fri, Apr 29, 2016 at 7:52 PM, Tanya Madurapperuma <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> *Introduction* >>>>>>> >>>>>>> The purpose of this feature is to provide a framework to generate >>>>>>> gadgets where you can plug datasource providers and chart templates. >>>>>>> >>>>>>> For an example, you will be able to plug your RDBMS datasource, rest >>>>>>> api , csv file , real time datsources etc as pluggable providers. >>>>>>> >>>>>>> *Flow* >>>>>>> >>>>>>> Select datasource provider (stage 1) --> Configure datasource >>>>>>> parameters (stage 2) --> Configure chart parameters (stage 3) --> >>>>>>> Preview >>>>>>> gadget (stage 4) --> Generate gadget (stage 5) >>>>>>> >>>>>>> *Proposed Architecuture* >>>>>>> >>>>>>> Provider developers can plug their providers to DS by adding their >>>>>>> respective provider into the designer.json >>>>>>> >>>>>>> *{* >>>>>>> * ............* >>>>>>> * "gadgetGeneration" :{* >>>>>>> * "isCreateGadgetEnable": false,* >>>>>>> * "providers": ["batch", "rdbms", "rest", "rt"]* >>>>>>> * },* >>>>>>> * ................* >>>>>>> *}* >>>>>>> >>>>>>> Can't we load this dynamically by scanning the files at server >>>>>> start? >>>>>> >>>>>> Provider implementation should be placed under >>>>>>> /portal/extensions/providers and should mainly contain 2 files. >>>>>>> >>>>>>> - config.json - contains the expected configuration in the >>>>>>> "Configure datasource parameters" stage 2 >>>>>>> >>>>>>> *example config* >>>>>>> * {* >>>>>>> >>>>>>> * "id":"rdbms",* >>>>>>> * "name" : "Relational Database Source",* >>>>>>> * "image" : "",* >>>>>>> * "config" : [* >>>>>>> * {* >>>>>>> * "fieldLabel" : "Database URL",* >>>>>>> * "fieldName" :"db_url",* >>>>>>> * "fieldType" : "text",* >>>>>>> * "defaultValue" : "",* >>>>>>> * "isRequired" : true* >>>>>>> * },* >>>>>>> * {* >>>>>>> * "fieldLabel" : "Username",* >>>>>>> * "fieldName" :"username",* >>>>>>> * "fieldType" : "text",* >>>>>>> * "defaultValue" : "",* >>>>>>> * "isRequired" : true* >>>>>>> * },* >>>>>>> * {* >>>>>>> * "fieldLabel" : "Password",* >>>>>>> * "fieldName" :"password",* >>>>>>> * "fieldType" : "password",* >>>>>>> * "defaultValue" : "",* >>>>>>> * "isRequired" : true* >>>>>>> * },* >>>>>>> * {* >>>>>>> * "fieldLabel" : " Database Driver",* >>>>>>> * "fieldName" :"driver",* >>>>>>> * "fieldType" : "dropDown",* >>>>>>> * "valueSet" : [ ],* >>>>>>> * "isRequired" : true* >>>>>>> * },* >>>>>>> * {* >>>>>>> * "fieldLabel" : " Check Box",* >>>>>>> * "fieldName" :"checkbox",* >>>>>>> * "fieldType" : "checkbox",* >>>>>>> * "isRequired" : true* >>>>>>> * },* >>>>>>> * {* >>>>>>> * "fieldType" : "advanced",* >>>>>>> * "partialName" : "myPartial"* >>>>>>> * }* >>>>>>> * ]* >>>>>>> *} * >>>>>>> >>>>>>> The configuration UI will be dynamically generated for the data >>>>>>> types text-box, checkbox, password field and drop-down. And a provider >>>>>>> can >>>>>>> plug his own UI blocks as partials when there are advanced fields. >>>>>>> >>>>>>> >>>>>>> - index.js - implements the provider api for >>>>>>> - validateData (providerConfig) >>>>>>> - To validate the inputs provided at the stage 2 >>>>>>> - providerConfig will be key value pairs of provided field >>>>>>> names and values >>>>>>> - getSchema (providerConfig) >>>>>>> - Returns the list of column names and their data types of >>>>>>> the configured data provider >>>>>>> - getData (providerConfig,schemaPropertyList) >>>>>>> - schemaPropertyList will be list of column names >>>>>>> selected at stage 3 >>>>>>> >>>>>>> >>>>>>> IMO the index.js should have a getConfigInfo() which will return the >>>>>> config.json which the UI will be plotted against, this is because for >>>>>> batch >>>>>> and realtime we have to pre populate the tables/stream. >>>>>> I think the validation of data can be part of >>>>>> getSchema()/generateSchema(). >>>>>> >>>>>> Similarly chart templates can also be plugged via configuring at >>>>>>> designer.json >>>>>>> Chart template mplementations also goes under >>>>>>> /portal/extensions/chart-templates and have 2 main files. >>>>>>> >>>>>>> >>>>>>> - config.json - contains the fields that needs to be populated >>>>>>> at stage 3 >>>>>>> >>>>>>> *example config* >>>>>>> >>>>>>> *{* >>>>>>> * "id":"lineChart",* >>>>>>> * "name" : "Line Chart",* >>>>>>> * "config": [* >>>>>>> * {* >>>>>>> * "fieldLabel": "X axis",* >>>>>>> * "fieldName": "x",* >>>>>>> * "fieldType": "String",* >>>>>>> * "fieldValue" : ["$COLUMN_NAMES"]* >>>>>>> * },* >>>>>>> * {* >>>>>>> * "fieldLabel": "Y axis",* >>>>>>> * "fieldName": "y",* >>>>>>> * "fieldType": "Int",* >>>>>>> * "fieldValue" : ["$COLUMN_NAMES"]* >>>>>>> * },* >>>>>>> * {* >>>>>>> * "fieldLabel": "Chart Colour",* >>>>>>> * "fieldName": "colour",* >>>>>>> * "fieldType": "String"* >>>>>>> * }* >>>>>>> * ]* >>>>>>> >>>>>>> *}* >>>>>>> >>>>>>> If the first element of field value is * ["$COLUMN_NAMES"] *,it >>>>>>> will populate the list of column names retrieved getSchema method of >>>>>>> provider api. >>>>>>> >>>>>>> >>>>>>> - index.js - implements the chart template api for >>>>>>> - draw (chartConfig, data) >>>>>>> - responsible for plotting the chart/gadget >>>>>>> - chartConfig will be key value pairs of chart >>>>>>> configurations provided at stage 3 >>>>>>> - data will be the actual data retrieved by calling >>>>>>> getData method of provider api >>>>>>> >>>>>>> >>>>>>> Here too we have to pass the schemaPropertyList to the chats >>>>>> index.js and generate the config.json, because for different type of >>>>>> charts >>>>>> we should be able to generate appropriate chat configs based on the >>>>>> schema, >>>>>> and validation of chart input can be done at the draw(). >>>>>> >>>>>> >>>>>>> Appreciate your input on the above approach. >>>>>>> >>>>>>> Thanks, >>>>>>> Tanya >>>>>>> >>>>>>> -- >>>>>>> Tanya Madurapperuma >>>>>>> >>>>>>> Senior Software Engineer, >>>>>>> WSO2 Inc. : wso2.com >>>>>>> Mobile : +94718184439 >>>>>>> Blog : http://tanyamadurapperuma.blogspot.com >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> *S. Suhothayan* >>>>>> Technical Lead & Team Lead of WSO2 Complex Event Processor >>>>>> *WSO2 Inc. *http://wso2.com >>>>>> * <http://wso2.com/>* >>>>>> lean . enterprise . middleware >>>>>> >>>>>> >>>>>> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog: >>>>>> http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: >>>>>> http://twitter.com/suhothayan <http://twitter.com/suhothayan> | >>>>>> linked-in: >>>>>> http://lk.linkedin.com/in/suhothayan >>>>>> <http://lk.linkedin.com/in/suhothayan>* >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Regards, >>>>> >>>>> Dunith Dhanushka, >>>>> Senior Software Engineer >>>>> WSO2 Inc, >>>>> >>>>> Mobile - +94 71 8615744 >>>>> Blog - dunithd.wordpress.com <http://blog.dunith.com> >>>>> Twitter - @dunithd <http://twitter.com/dunithd> >>>>> >>>> >>>> >>>> >>>> -- >>>> Tanya Madurapperuma >>>> >>>> Senior Software Engineer, >>>> WSO2 Inc. : wso2.com >>>> Mobile : +94718184439 >>>> Blog : http://tanyamadurapperuma.blogspot.com >>>> >>> >>> >>> >>> -- >>> Tanya Madurapperuma >>> >>> Senior Software Engineer, >>> WSO2 Inc. : wso2.com >>> Mobile : +94718184439 >>> Blog : http://tanyamadurapperuma.blogspot.com >>> >> >> >> >> -- >> Tanya Madurapperuma >> >> Senior Software Engineer, >> WSO2 Inc. : wso2.com >> Mobile : +94718184439 >> Blog : http://tanyamadurapperuma.blogspot.com >> > > > > -- > Tanya Madurapperuma > > Senior Software Engineer, > WSO2 Inc. : wso2.com > Mobile : +94718184439 > Blog : http://tanyamadurapperuma.blogspot.com > -- Regards, Dunith Dhanushka, Associate Technical Lead WSO2 Inc, Mobile - +94 71 8615744 Blog - *https://medium.com/@dunithd <https://medium.com/@dunithd>* Twitter - @dunithd <http://twitter.com/dunithd>
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
